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ABSTRACT 


Computer networks are the outcome of the combination of 
computer and data communications technologies. Part I of 
this thesis presents the background of each one of these 
technologies and exposes analytically the principles of 
computer networking in particular. In Part II, a preli- 
minary design is developed for the Naval Postgraduate School 
(NPS) time sharing system. The study of Part II leads to 
a comparative analysis of four network architectures which 
are suitable for satisfying the needs of the NPS. Those 
architectures are compared in view of the desirable attri- 
butes of accessibility, evolvability, modularity, reliability 
and eas of operation. The communications requirements are 
derived and technologies for implementation of the communi- 


Cations links are recommended. 
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I. INTRODUCTION 


Computer systems can be viewed in two ways: as informa- 
tion systems or as computation systems. In an information 
system data manipulation, data storage and retrieval in 
databases predominates over the computation aspect. Ina 
computation system the mathematical transformation of input 
data into output is the main purpose of the system. Both 
aspects coexist in any computer system, yet generally one is 
stronger than the other. 

Modern society is dominated by the existence of big and 
complex organizations inside and outside of governments. The 
only way to achieve efficiency, coordination and control in 
these glants is through prompt and reliable information. On 
the other hand, technology and science has progressed toward 
a sophistication which requires more computer power. 

The greatest motivation for computer networks is, regard- 
less of the main purpose of the computer system, sharing of 
expensive computers. The resources that can be shared are 
databases, processors and software. The sharing of data 
bases is the more important aspect for information systems, 
while the sharing of expensive, powerful processors, special- 
ized processors and software is the main goal of computation 
systems. By sharing those expensive resources, computer 
networks can achieve cost-effectiveness. 


Other important motivations for computer networks are 


1Lae 








mecessibility, flexibility, generality, availability, relia- 
bility and efficiency. 

Those points are best stressed by the following quota- 
fron of Dr. Ruth M. Davis* 

"1. Computer networks are essential for all those real 

time geographically dispersed control activities vital 

to our individual and national well-being. 

2. Computer networks are the only practical means 

available for the sharing of expensive information re- 

sources, computing resources, and information handling 
equipment. 

3. Computer networks are the only practical means of 

providing equality of access to and an equality of 

quality in public services, independent of geographical 
location. 

4, Maxicomputers available through computer networks are 

perhaps the only economically justifiable means for the 

large scientific calculations essential to the advance- 
ment of much needed basic research and engineering. 

5. Centralized management, in a real-time sense, of geo- 

graphically dispersed organization is impossible without 

computer networks." 

This thesis is divided in two parts. Part 1, Computer 
Networks, comprises chapters II to VI and is an analytical 
exposition of the technology and principles of computer net- 
' works with emphasis on generality and breadth. The second 
- part, Design of a Network for the NPS Time Sharing System, 


comprises chapters VII to X and is a study leading to the 


Suggestion of network architectures which are advantageous 





Computing Networks: A Powerful National For 
| bad J, a es 
Speech COMPCON 73, February 27, 1973. e-Keynote 





for the replacement of the present time sharing system at 


the Naval Postgraduate School. 


I 





PART 1 


Computer Networks 
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II. COMPUTER NETWORK TECHNOLOGY 


A computer network, also called computer-communication 
network, is, in the broad sense, any system composed of one 
or more computers and terminals, communication transmission 
facilities, and specialized or general purpose hardware to 
Facilitate the flow of data between terminals and/or pro- 
cessors. Its parts consist of communication devices, host 
processors, transmission lines and a set of rules, implemen- 
ted in either hardware or software, to insure the orderly 


flow of traffic in the network /Ref. 9/. 


A. COMMUNICATION CHANNELS 

A communication channel is a path along which signals or 
data can be sent. 

From the point of view of the user a discrete channel, 
1.e., a channel used to transmit a finite set of symbols, is 
measured by its capacity or transmission rate and its pro- 
Dability of error. The capacity is the Speed at which infor- 
mation can be transmitted by the channel in symbols/second 
and usually given in bit/sec (binary digits/second). The 
probability of error or error rate of a channel measures the 
degree of uncertainty associated with the transmission of 
information over the channel and is sometimes given in bits 
in error/bits transmitted (probability is a dimensionaless 
number); it is generally referred to as the bit error rate 


Or, BER. The probability of error of a channel may be made 
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as small as desired; one of Shannon's theorems states that 
it is possible to transmit information through a channel up 
to its capacity C with an arbitrarily small frequency of 
errors and that it is not possible to transmit at a rate 
megher than C with an arbitrarily small frequency of errors. 
This result may be regarded as a theoretical possibility; it 
is, indeed, practically possible to reduce the probability 
of error of a channel as much as desired by the use of error 
correcting codes, but at the cost of diminished capacity in 
the transfer of information; so, capacity can be traded for 
error rate. Up to now, no method is known for constructing 
codes that can achieve data speeds near capacity with van- 
ishingly small error rates. 

A continuous channel, that is, a channel which can trans- 
port analog signals or continuous waveforms, can be charac- 
terized in a very simplified way by two main factors: its 
frequency band and signal-to-noise ratio. “hve frequency 
band measures the capability of the channel in transporting 
Sinusoidal signals of different frequencies; it can be con- 
cisely (although not precisely) specified by the lowest and 
highest frequencies which are 3 dB (half-power) below the 
center of the band of the channel transfer characteristic. 

A more simplified measure is the bandwidth of the channel 
which is the difference between the maximum and minimum 
frequency of its frequency band. | The Signal-to-noise ratio 


1S a measure of the purity of the signal which arrives at 
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the receiving point. 
The capacity of a continuous channel in bps is related 
to its bandwidth W and signal-to-noise ratio S/N by another 


one of Shannon's Theorem /Ref. 42/: 
C = Wlog are) 
2 N 


Bodimy tnisc is a theoretical limit. In practice, the 
capacity of a continuous channel given by the equation above 
is far from being achieved. 

Many times the term "channel" is used implying one way 
communication, whereas the word ee eee eee implies 
two-way communications. Usually a communication circuit 
is defined as the complete electrical path providing one or 
two way communication between two points comprising associa- 
ted wena and receive channels. 

Whether or not a communication facility has the capabil- 
ity of transmitting in both directions, simultaneously or 
not, can be an important factor in system design. According 
to this capability the operation of a communication facility 
may be classified into /Ref. 1/7: 

1. Simplex operation - only one direction of transmission; 

2. Half-duplex operation - two directions of transmission 


are used one at a time; 


3. Full-duplex operation - both directions of trans- 


mission can be used Simultaneously. 
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Some communication media have the ability of simultan- 
eous multipoint connection. Broadcast systems are those 
which permits simultaneous transmission from a single source 
to a variety of recipients. The predominant operational 
characteristic is that only one source can be transmitting 
at a time, while many other components of the system may be 
receiving from that source at the same time. 

A point-to-point communication media, on the other hand, 
permits transmission and reception only between clearly de- 


fined termination points. 


B. POINT-TO-POINT COMMUNICATIONS 

Establishment of point-to-point communications 1s one 
subject of the communication engineering field. The tech- 
nology for implementation of point-to-point circuits varies 
according to technical, economical and systems factors and 
can be classified into two groups: transmission lines and 
radio links. 

Transmission lines are physical paths which can carry 
Or guide electrical or electromagnetic waves. Examples of 
transmission lines are twisted pairs of copper wires, coaxial 
cables, wave guides, open wire, parallel-wire lines and fiber 
Sotics. 

When it is necessary to transport information through a 
communication system this information is first put into the 


form of an electrical signal. Usually this signal cannot 
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be transmitted in its original form; this is possible through 
some transmission lines, usually in short distances; it is 
almost never possible through radio links. A intermediate 
transformation has to be performed. This original signal 
that is to be transmitted is called the baseband or basic 
Signal. A sinusoidal signal called the carrier has one or 
more of its attributes of phase, frequency and amplitude 
modified by the baseband eenal through the process of modu- 
lation and then is fed in the transmission media. Thus, 
through modulation the baseband signal is superimposed on 
the carrier, which transports or "carries" it. Depending 

on what atribute was altered, the process is called ampli- 
tude modulation (AM), frequency modulation (FM) or phase 
modulation (PM). The converse process of extracting the 
baseband signal from the carrier is called demodulation. 
This process is accomplished by sensing the attribute of the 
Carrier which was previously modified. 

Radio links utilize the phenomenon of electromagnetic 
‘propagation through a media, usually the atmosphere, to 
transport information. The frequency of the carrier deter- 
mines important characteristics of the system. The most 
suitable bands of the radio spectrum for data transmission 
in computer networks (and in general) are the high end of 
VHF band (30-300MHZ), UHF (300-3000MHZ) and SHP (3-30GHZ). 
Because of problems of congestion of the spectrum, the VHF 


. band is seldom used in computer networks. 
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Due to the cost involved, rarely is a communication system 
built for the implementation of a geographically dispersed 
computer network. Instead, to minimize the initial invest- 
ment, communications facilities are obtained from common- 
carriers: Common-carriers are public utility companies that 
are recognized by an appropriate regulatory agency as having 
the authority and responsibility to furnish communication 
services to the general public. 

For example, for great distances and small capacities it 
is more economical to lease a point-to-point circuit from 
a common-carrier. For short distances and large capacities 
it may be advantageous to implement a point-to-point communi- 
cation facility over the option of leasing, if the costs are 
computed over the life cycle of the svstem. Typically, trans- 
continental lines are leased and intra-facility lines are the 
user's property. 

Data services offered by common-carriers can be divided 
into private (leased) lines and switched (public) lines 
/Ref. 1/7. In either case there is a variety of bandwidths. 
For voice bandwidth facilities the switched data calls are 
routed through the ordinary DDD (direct distance dialed) 
telephone network and a different connection is usually ob- 
tained each time a call is dialed. On the other hand, a 
private line does not traverse the Switching equipment and 
its transmission characteristics are fixed. Conditioning is 


the technique of compensating for undesirable transmission 
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characteristics on a leased line by the use of equalization 
filters. This provides greater bandwidth and transmission 
quality for data transmission than a switched connection. 
Depending on the length of the line and usage pattern, the 
cost of a private line can be more or less than the cost of 
the switched network. Typically, if more than several hours 
of traffic are to be carried each day between two points, 
the private line is the more economical of the two alterna- 
tives. On the other hand, voiceband switched lines are used 
where the delay in establishing the connection is not impor- 
tant and where a slightly higher probability of error can be 
tolerated. Switched lines are typically used in computer- 
terminal connections. 

The bandwidth of a voicehand circuit is something less 
than 3KHZ. These circuits can support data rates of up to 
4800 bps on DDD calls and up to l0kbps on private lines 
pRef. 1/. 

Narrowband circuits are those of less than voice band- 
width, obtained by division of a voiceband facility into 
smaller ones. 

Wideband circuits are those of greater width than a 
Single voice bandwidth. There is a hierarchy of wide-band 
channels, obtained by the aggregation of smaller ones. 

Above the voice bandwidth comes next the group, which is 
equivalent to 12 voice Channels, has 48 KHZ of bandwidth and, 


with associated modulation and demodulation devices, gener- 
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ally transmits data at a 50 kbps rate, although higher speeds 
are possible /Ref. like Nearly all wideband circuits are pri- 
vate line arrangements, but there are switched networks avail- 
able for broadband transmission. Occasionally wideband cir- 
cuits (50 Kbps) are leased from a common carrier and used in 

a private switched network, such as the Department of Defense 
ARPA network. 

The common-carriers, in addition to the analog channels 
primarily designed for transmission of voice, also offer true 
digital transmission systems, as, for example, the T-l carrier 
which has a capacity of 1.544 Mbps and may transport up to 24 
voice channels in digital form, using pulse code modulation 


(PCM) /Ref. 1/. 


C. SWITCHING 
The number of communication circuits required for di- 

rectly connecting any pair of a network with N stations is 
equal to oe Thus, as the number of points increases, 
the option of complete interconnection becomes economically 
infeasible. For this reason, large networks using point-to- 
point communication facilities are not completely connected. 
To share the existing circuits in a manner that allows com- 
munication between any pair of points, switching techniques 


are employed. 


marcuit Switching, as illustrated by the voice telephone 


network, is a system whereby a complete physical path is 
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established from sender to receiver that remains in effect 
for the duration of the conversation /Ref 10/7. In most 
telephone exchanges, this route is actually established by 
mechanical means (such as relays), though the selection of 
fhe route may be made electronically. The process of select- 
ing a route, or call establishment may take on the order of 
seconds for a complex network. This speed is usually deemed 
too slow to be effective for supporting dynamically estab- 
lished communication between host computers. However, the 
increasing speeds associated with improved switches have 
permitted utilization of circuit switching by computer 
communications utilities. On the other hand, once the route 
is established, data transfer is continuous through the 
network in the sense that there are no delays added to data 
by the switches. End to end transmission delay is determined 
by the propagation times of the various circuit media em- 
ployed. Circuit switching has been the primary technology 

in support of computer-terminal connections in remote-access 
networks. 

Effective utilization of circuit switching requires care- 
ful matching of circuit capacity against the transmission 
requirements, which usually are time-variant. Unused band- 
width cannot be shared with other pairs of network components 
and the maximum information transfer rate is limited by the 
Capacity of the circuits in the route. For these reasons, 


Circuit switch seems ill-suited to Slits VEL aih ie. asin 
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interactive use, because throughput requirements vary and re- 
sult in line under utilization. Another factor is that ein — 
cuit switching is subject to line contention delay when some 
circuits required for a particular path are busy. 

A message is defined to be a logical unit of information 
from the viewpoint of the user. Telegrams, files, programs 
and queries are examples of messages. As a natural alterna- 
tive to circuit switching, a system providing message de- 
livery service can be devised. Thus, in a role analogous 
to the Post Office, a message is "dropped into" the system 
which assumes responsibility for its delivery. In message 
switching, an individual message is stored and forwarded at 
each switch, along its path from source to destination, on 
the basis of address information it carries with it. Message 
Switching differs from circuit switching in that circuits are 
not dedicated for exclusive use of given sender-receiver pair. 
As a result, the opportunity exists for utilizing one cir- 
cuit to carry traffic between several source destination 
pairs. This, in turn, permits increased circuit utilization 
/Ref. 297. 

Two challenges in achieving effective operation of a 
message switched system must be resolved: (1) balancing re- 
Source utilization against delay; and (2) introducing some 
type of mechanism to give short messages reasonable service 
meet. 297. 


Transmission delay between source and destination exists 
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in message switching as a result of the contention for re- 
sources one message encounters each time it reaches a switch 
and waits for transmission to the next step. The first issue 
is a typical management problem; the conflicting desires of 
users and management must be mediated. Users want minimum 
delay, which requires low utilization, whereas management 
demands high resource utilization, which incurs long delays. 
The second problem can be solved by having the quality of 
service proportionally distributed according to the amount 
of work requested i.e., manage the system in such a way that 
the delays are approximately proportional to the size of the 
messages. One method of accomplishing this objective is to 
establish a priority scheme based on the message length. 

But this can conflict with operational priorities. Moreover, 
it is natural to attempt to defeat this mechanism by sub- 
dividing a long message into shorter pieces. 

Packet switching is a special type of message switching 
distinguished by a number of characteristics which define a 
particular type of communication system. With this techni- 
que, messages are typically broken into a series of short, 
fixed-length, addressed packets of data which are routed 
independently to their destination using store~-and-forward 
procedures as in message-switching /Ref. 107. At the destin- 
ation message processor, the message is reassembled from the 
Packets. Single packet messages can then be transmitted with 


a minimum of delay while throughput can be achieved for multi- 
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packet messages through simultaneous EEahotroclOl on Several 
packets through different paths. Through a complex evaluation 
of individual circuit and switch loads, packet switching 
Gomputers dynamically alter routing of packets to the mini- 
mum delay circuits. Thus, packets of one message may tra- 
verse different routes and arrive at the destination out-of- 
ercaer. 

Packet-switching networks typically implement a "logical 
Circuit" (in no way related to the physical paths traversed 
by packets) from source to destination with the following 
properties /Ref. 17/: 

l. high availability-achieved by means of the routing 
algorithm with the capability of alternate paths; 

2. fluctuating delay-due to the statistical nature of 
the switching process; 

=. stluctuating data rate; 

4, packet transmission speed advantages (relative to 
message switching) are achieved at the cost of control over- 
head on each data packet transmitted. 

Packet switching networks reduce the delay associated 
with message-switching systems by eliminating secondary- 
Storage buffering of data, needed in message-switching be- 
Cause of the space required for holding a queue of messages. 
In addition, data is transmitted as small units (packets) to 
reduce queuing on the SESE CLECUTES and al low “pipelining” 


from source to destination as mentioned above. 
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Compared to circuit switching, packet switching 1s more 
effective if most of the messages are short as in data base 
query/response and interactive traffic. But circuit switch 
is better if all messages are "long", e.g., file transfer. 

In multi-modal traffic, i.e., mix of long and short messages, 
packet switching appears to be slightly more advantageous 


/Ref. 29/. 


D. MODEMS 

The utilization of the widespread telephone network is, 
from the point of view of the user, practical and economical 
for the transmission of data. However, the telephone network 
Gonsists of voiceband channels. To transmit digital signals 
over these analog channels, which pass frequencies in the 
range 300-3400 HZ, it is necessary for a data transmitter to 
Modulate a voice-frequency carrier signal and for a data 
receiver to demodulate this signal. A data transceiver is 
consequently known as a MODEM. Such a modem serves to inter- 
connect data equipment with communication circuits. [In 
addition to its basic function of translating between the 
binary digital signals of the data equipment and the modu- 
lated voice-frequency signals of the communication channel, 
it also performs a number of control functions which coor- 
dinate the flow of data between the data equipment. 

The majority of modems are designed to accept and de- 
liver a serial stream of binary data at the data equipment 


interface. There are, however, some types of modems which 
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handle a character at a time by accepting and delivering 


binary signals on several parallel interface leads and which 


modulate multiple voice-frequency carriers for parallel 
transmission over the voice channel /Ref 1/. 

One common type of modem is the acoustic~coupled modem, 
in which a standard telephone handset is inserted into an 
acoustic interface connected to the modem. Acoustic-couple 
modems operate reliably to speeds of about 300 bps, but are 
very much used because of their portability (no need for 
electrical connection to the telephone network). 

Private-line modems are available for four-wire, full- 
duplex operation. In modems intended for two-wire operatio 
a reverse channel for slow-speed signaling in the reverse 
direction is usually present. Full-duplex operation in two 
wire channels is only possible at low data rates. [In half- 
duplex operation the time required to turn the direction 
of transmission around becomes important. Also related to 
this turn around time is the initial acquisition period, i. 
the delay between the request for transmission and the re- 
ception of permission to transmit. This set-up time can 
amount to several hundred milliseconds. In polled systems 
this delay can become important. 

In many high-speed modems a so-called automatic eCquali- 
zation is used to cope with amplitude and delay distortion 
and to reduce line conditioning requirements /Ref. Ve 


Automatic or adaptive equalization is implemented using 
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transversal filters, which are tapped delay lines with ad- 
justable gain at each tap feeding a summing amplifier. The 
gains are set by feedback circuitry. One drawback of auto- 
Matic equalization is the necessary extension of the modem 
set-up time to approximately adjust the tap settings with 
special test signals. However, automatic equalization can 
double the data rates achievable on the telephone channel. 

There are modems for synchronous and asynchronous opera- 
tions. In asynchronous mode one character is sent at a time, 
preceeded by a start bit and terminated by a stop bit. In 
the synchronous mode characters are sent in a continuous 
stream. Transmission 1s synchronized by a clock internal to 
the modem with a high degree of stability and precision. 
This fixed rate of transmission does not require start and 
Stop bits. Almost all high-speed modems are synchronous. 

Binary frequency modulation in the form of frequency 
shift keying (FSK) is the usual choice when Simplicity and 
economy are more important than bandwidth efficiency /Ref. ae 
Typically one frequency is selected to represent the zero 
and another frequency is selected to represent the one. 

Phase modulation (PM) in the form of differential phase- 
shift keying (DPSK) is employed for higher speeds /Ref 1l1/. 
The term differential implies that symbol meaning is based 
On a change in phase from the previous state and not an 
absolute phase reference. 


Amplitude modulation (AM) is used in LionoMiietilmag ima ta — 
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level symbols, for achievement of high data rates, parti- 
cularly in connection with a reduction of bandwidth technique 
such as vestigial side band (AM-VSB) /Ref. 11/. 


Typical data rates for switched lines are /Ref. 1/: 


JA 


300 bps asynchronous 


[A 


1200 bps asynchronous 

2400, 3600, 4800 bps synchronous; 

and for private voiceband circuits are: 
< 1800 bps asynchronous 


2400, 4800, 7200, 9600 bps synchronous. 


me. MULTIPLEXING 

Communication lines are a major component in the cost of 
a geographically dispersed computer network. Thus, minimi- 
zation of communication lines cost is an important objective 
in network design. Multiplexing is one way of diminishing 
the cost of communications. Multiplexing refers to the 
technique of aggregating a number of “low" speed signals into 
a "high" speed signal. Then, instead of using a low capacity 
Channel for each "low" speed signal, a high capacity channel 
Carries the aggregated signal. At the other end of the 
Channel the aggregated signal is demultiplexed, i.e., it is 
Separated into its "low" speed components. The process is 
completely transparent to the user. Figure 2.1 illustrates 
the multiplex process. 


The economical reason behind multiplexing is that the 
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relation cost/bandwidth (dollar per hertz) for leased lines 
decreases as the bandwidth (or capacity) of the line in- 
creases. Then, it is cheaper to lease one high capacity 
line than many low capacity lines with the same total capa- 


city. Multiplexing is extensively used in the telephone 
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Figure 2.1 - Multiplexing 


network for voice channels; even disregarding economics, 
without multiplexing it would not be possible to implement 
the multiplicity of the long-haul circuits in existence to- 
day over microwave links, because of the problems which a 
great number of radio frequency carriers would Cause in the 


utilization of the frequency spectrum. 
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There are two techniques used for multiplexing: frequency 
division multiplexing (FDM) and time division multiplexing 
(TDM) . 

mn frequency division multiplexing the frequency band- 
width of the composite channel is divided into smaller bands 
which are allocated to each component channel (subchannels). 
A typical FDM system for data transmission is shown in Figure 
2.2 /Ref. 147. In this example, each subchannel is assigned 
to a frequency band where the binary digits are transmitted 
in FSK, i.e., one tone corresponding to 1 and another 
corresponding to 0. Another modulation process could be used 
in the subchannels such as PSK, DPSK, etc., but FSK is 


generally used for reasons of simplicity and economy. 


Tones for binary’ digits 


Freq. 
SC 1 SC 2 SC 3 SC 4 


Total bandwidth 





SC — subchannel! 


Figure 2.2 - Spectrum partitioning in FDM 
multiplex for data transmission 


32 





When FDM is used on a voice grade line, each subchannel 
may typically transmit data asynchronously at speeds up to 
150 bits/s, although in special cases at faster speeds. One 
of the limitations of FDM stems from the need for guard bands 
or safety zones between adjacent subchannels to prevent 
mutual interference /Ref. 14/. These guard bands impose a 
practical limit on the efficiency of an FDM system. Also, 
the generally used FSK 1s not the better mocueion process 
for bandwidth efficiency. For example, with an FDM equip- 
ment operating on a private voice grade line, the maximum 
composite or aggregate low speed bit rate achievable will 
typically range from 1800 to 2000 bits/s, although in some 


cases it can be slightly higher /Ref. 14/. 





C- COMPUTER 
M- FOM MULTIPLEXER T- TERMINAL 
Figure 2.3 - Derivation of subchannels with FDM 
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FDM's primary advantage to end users is its relatively 
low cost in applications where FDM's aggreagte bit-rate limit 
is not constraining. Another advantage of FDM is the ease of 
making subchannels derivations at intermediate points in a 
route as exemplified in Figure 2.3 /Ref. 14/. Thus FDM is 
particularly cost-effective in multiplexing an unclustered 
terminal group whose aggregate bit rate does not exceed the 
above mentioned limit. 

TDM can be performed in two ways, as synchronous time 
division multiplexing (STDM) or as asynchronous time division 
Multiplex (ATDM). 

STDM aggregates incoming "low" speed channels by allocat- 
ing a time slot for each one in the composite frame of the 
"high" speed output. Thus, as shown in Figure 2.4 the 
relative position of the time slot inside the frame determines 
to what subchannel the information belongs. STDM samples 
each incoming subchannel at a constant rate and order, peels 
off bits or characters and interleaves them onto the high- 
speed data stream. 

For data communications STDM is generally more efficient 
than FDM because it is capable of higher speeds over the same 
bandwidth. The example, while FDM goes up to 2000 bps over 
Private voice grade lines, STDM can operate at speeds of 4800 


f 


7200, and even 9600 bit/s in certain instances /Ref. 147. 
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Figure 2.4 - STDM Principle 
STDM equipments have to be cascaded to modems for connec~- 
tion to common-carrier lines because their outputs and inputs 
are digital signals. FDM's, in contrast, don't need modems 
because they transmit and receive analog signals. A typical 


Muse of STDM is shown in Figure 2.5. 


STDM may perform bit or character interleaving on the 
Shared line when serving Start-stop (asynchronous) terminals 
exclusively. In these applications, character interleaving 
is usually more efficient since a modest amount of bandwidth 


Compression is possible. The start and Stop bits of each 
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COMMON-CARRIER 


CIRCUIT j" 


LONG DISTANCE 
C - COMPUTER TERMINAL 
M- MODEM 
Figure 2.5 - Connection of terminals to a 


computer with STDM 
character entering the STDM may be stripped off prior to the 
character's insertion into the frame of multiplexed data. 
Bits stripped from incoming data characters are reinserted 
by the demultiplexing unit prior to the distribution of the 
Characters to their respective output lines. Thus, the in- 
coming characters are effectively reencoded using fewer bits 
per character for transmission over the shared line, enabling 
a typical aggregate low-speed bit rate of 1.1 times the 
Shared link's transmission rate /Ref. 1, 14/7. 

When used in conjunction with synchronous lines, STDM 
generally uses bit interleaving. Whether bit or character 
interleaving is used, special predetermined code sequences 
are used between STDM's to define the beginning of each new 
frame of multiplexed data and to maintain synchronization. 
Demultiplexing is thus accomplished assuming an implicit re- 


lationship between the output lines and the relative position 
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of the time slot as shown in Figure 2.4. 

In comparison to FDM's, STDM's are expensive for deriva- 
tions because a relatively complete STDM unit and associated 
modem have to be used at any point where one or more sub- 
channels are being picked up or dropped. Also, the problem 
of synchronization among the multiples STDM units has to be 
addressed. Fundamentally two options exist in this regard - 
to use a master clock from which all elements derive their 
timing or to use an independent synchronous clock at each 
line. This second process requires buffering to absorb a 
data build due to slight variations in the frequency of in- 
dependent clocks. Most of the STDM networks use the master 
clock option due to economic reasons /Ref. 147. Figure 2.6 
displays an example of a synchronous TDM network with de- 


EivVations. 
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Figure 2.6 - STDM network with derivation 
In summary, both FDM and STDM are widely used in data 
Networks for economy in communication costs. The main usage 
has been in computer-remote terminals connections. When the 
aggregate low-speed bit rate for all terminals does not 


exceed 2000 bps, either FDM or STDM can be used, but FDM 
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will be more cost effective /Ref. 14/. Whenever a higher 
aggregate bit rate is required, or whenever any synchronous 
terminals are included in the sharing group, STDM will 

usually be required. However, in higher bit rate applica- 
tions involving geographically diverse terminal locations, the 
use of an integrated blend of FDM and STDM is indicated. FDM's 
are used to span isolated terminal sites, creating traffic 
clusters which are then synchronously multiplexed to one or 
more distant computer sites. 

Asynchronous time division multiplex (ATDM) differs from 
STDM in that a dedicated subchannel is not provided for each 
terminal in the sharing group. Since, under certain condi- 
tions of heavy loading, an ATDM may be incapable of accomo- 
dating all incoming lines in the shared channel, statistics 
and queuing becoming important considerations. Thus, it is 
a hybrid technique between multiplexing and concentration 
(that will be discussed later) and is often called statistical 
multiplexing. 

The fundamental notion behind ATDM is to exploit the fact 
that in STDM systems, many of the time slots in the fixed- 
format frames are wasted since a typical remote terminal will 
actually be transmitting data less than 10 percent of the time 
it is on line. As shown in Figure 2.7, ATDM dynamically 
allocates the time slots in a frame of data to the currently 
active users, reducing the fraction of wasted time slots and 


thereby increasing overall line utilization and throughput 
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/Ref. 1, 12, 147. Analytical studies indicate that approxi- 
mately from 2 to 4 times as many users could be accomodated 
on a voice grade line as with STDM, assuming an application 
environment where either method could be used /Ref. 1, 12/. 
In certain situations where low-duty cycle terminals are 
serviced by an ATDM system over a broad-band link, the mar- 


gin could be greater. 
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- Figure 2.7 - Comparison between STDM and ATDM 
The tradeoff disadvantages of ATDM are the costs of 
Substantially more elaborate addressing and control circuitry, 
data buffers to hold incoming messages, and the possibility 


Of blocking and queuing delays under heavily loaded condi- 


tions. 


Sg 








The complexity of control and the storage requirement for 
buffering lead to implementations of ATDM with the minicom- 
puter (Ref. 9/7. This minicomputer implementation implies a 
fundamental component cost of approximately ten times Ehe cost 
of a FDM or STDM unit. Such a figure requires large econo- 
mies to be achieved for cost effectiveness. Such economy 
can be achieved by handling a large number of channels which 
can go typically to 64 with increased cost /Ref. 9/7. Also, 
the idle capacity of the minicomputer can be used to perform 
local operations such as polling, error checking,’ line con- 
trol, etc., thus alleviating the central computer of those 


tasks. 


F., CONCENTRATION 

The word concentration appears to have a very broad 
meaning in data communications /Ref. ie he 

Sometimes the word multiplexing is restricted to FDM and 
STDM. In this context, concentration is distinguished from 
Multiplexing by the characteristic that the shared line or 
the high speed side of a device performing concentration has 
less capacity than the sum of all the capacities of the lines 
Which share the high speed facility (low speed side)/Ref. 1/7. 
In a multiplexer the sum of capacities at the low speed side 
nequals" the capacity of the high speed side. Thus, a multi- 
plexer is transparent to the operation and a concentrator not. 
ATDM according to this interpretation is classed as concentra- 


iON. 
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Sometimes concentration is used in a general sense encom- 
passing multiplexing and message concentration. According 
to this interpretation, concentrators may be of three types 
/Ref. ey 

1. Nonbuffered concentrators - those devices in which 
data are multiplexed by bit and don't buffer at a character 
level or more; FDM and bit interleaving STDM are in this 
class. 

2. Buffered concentrators - those devices that buffer 
at a character level before forwarding; one or more charac- 
ters are assembled as they are received, before forwarding 
at a higher rate; character interleaving STDM and ATDM are 
in this class. 

3. Store-and-forward concentrators - those devices that 
are 'block'-or 'message'-oriented, rather than character 
Oriented as above. 

In another usage concentration is used for message switch- 
ing concentration and line or circuit switching concentration 
meef. 14/. 

Message switching concentration (MSC), called store-and- 
forward concentration, involves the "Multiplexing" of entire 
messages or fixed-length portions of long messages. The MSC 
accumulates message blocks in it buffer until one is com- 
pletely assembled and the high-speed line is available to 
transmit it. Thus the high-speed line transmits variable- 


length frames of data with appropriate addressing and control 
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information; all data characters in each frame are generally 
associated with the same source-destination pair. MSC is 
thus a special case of message switching. 

The primary disadvantage of MSC, compared to FDM or STDM, 
are economic in nature, and relate to the cost of a stored 
program computer and buffer storage usually required /Ref. 

9, 147. Again, the use of a minicomputer has some supple- 
mentary benefits such as the capability of performing remote 
line control, code conversion, error checking, selective 
routing and error control of the multiplexed circuit. Some 
problems also arise when MSC's are required to handle 
messages with widely varying block lengths. As in message- 
Switching, when one extremely long user data block is 
assembled in the concentrator, it may tie up the shared line 
for an inordinately long time, at the expense of other users. 
A final disadvantage of MSC relates to its reliability 
Characteristics. Most individual component failures will 

put the system down, since minicomputers are used. The con- 
centration duplication is often used at each concentration 
node /Ref. 9, 36/7. The high costs generally make other line- 
sharing schemes more cost-effective in applications involving 
Moderate traffic. But, in large, multiuser networks, message 
Switching concentration affords flexibility and performance 
advantages over other line-sharing schemes /Ref. ve 

Circuit switching concentration involves a switching de- 


Vice which electrically bridges a PSCUp ROL me Inputs LO. a 
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group of m output links on a demand basis (n is typically 
from 3 to 5 times the value of m in commercial applications) 
/Ref. 147. Thus, circuit switching concentration is a 
special case of circuit switching. Ordinarily, the input 
lines and the output trunks to which they are switched have 
similar bandwidth and transmission properties. Private 
automatic branch exchanges (PABX) are examples of circuit 
Switches. Although they have been used for conventional 
voice telephony, such devices may function equally well when 
used for computer communications as a line switching con- 
centrator with the advantage that they may be built inex- 
pensively. 

No message queuing delays are introduced at the circuit 
Switch once a connection is established. But there 1s con- 
tention for the use of the trunks and several possibilities 
exist for allocating the output trunk lines. The serving 
mechanism may be first-come-first-serve, with requests that 
arrive at the switch when all trunks are occupied being lost. 
Another more complex scheme may queue incoming requests that 
were not satisfied for future serving, when trunck become 


available. 


The Figure 2.8 shows a typical application for a circuit 


Switching concentrator. 
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e Figure 2.8 - Example of circuit switching 
concentration 
G. COMMUNICATIONS PROCESSORS 

Communication processors, also called network processors, 
are Minicomputer based devices performing one or more of the 
following functions /Ref. 36/7: 

ies tLront-end processing; 

2. message switching; 

3. message concentration; 

4. terminal multiplexing and control. 

The front-end processor or FEP is a rather flexible con- 
cept so that, like many other computer terms, this one has 
no precise meaning. In general, a computer attached to a 
host computer and handling communications functions for this 


host is called a FEP /Ref. 127. 
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Many of the tasks performed in handling communications 
circuits are relatively simple, but highly repetitive, and 
can make considerable demands on the time of the host com- 
puter, which sometimes does not have appropriate capabili- 
ties for the real-time demands presented by data communi- 
cations. It often proves more economic to have these func- 
tions being performed separately by the front-end processor. 
The main advantages of this approach are /Ref. 12/: 

1. The cost of hardware to attach lines is often less 
with a minicomputer, because they are more flexible in 
accomodating a wide spectrum of communications network 
facilities, devices and line control procedures. 

2. The processing load removed from the main computer 
will considerably increase the power available for computa- 
tional purposes. 

3. It becomes possible to separate the complete system 
cleanly into two parts: the main processor and the communi- 
cations network. This gives increased flexibility and may 
allow one part to be enhanced or replaced without affecting 
the other. The communications network requirements are con- 
tinuously changing due to the need to accomodate a large 
number of different terminals and communications haedeineies < 
such changes should not be allowed to reflect back into the 
Yather complex host computer Operating system. With the use 
Of an FEP the desired isolation can be attained and any 


\ 


Configuration changes can be made in a smaller operating 
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environment. 

Various general purpose communications functions are re-~ 
quired by almost every application. These functions, which 
are normally handled by software, include /Ref. 36/7: control 
of the communications network and its terminals, buffering 
and queuing of messages, checking message validity, authori- 
zation of the message senders and receivers, routing of 
messages between their different sources and sinks (as re- 
presented by different programs and/or terminals), message 
editing and formatting, etc. The partitioning of these 
functions between the FEP and the host computer may be done 
in various degress. More benefit of the use of the FEP will 
be achieved as the FEP assumes more communications functions 
thus alleviating the host of this load. 

The connection of a FEP to a host may take different 
forms. In the simplest one the FEP replaces and emulates a 
communications controller. The FEP is thus attached to the 
host or to the host channel and acts as a device controller. 
Such a FEP is sometimes called transparent because it gives to 
the main computer a 'familiar' picture of the terminals. The 
advantage in this case can be its low cost compared with the 
Stardard equipment, but it has the special advantage of flex- 
ibility, i.e., it can be made to accept 'foreign' terminals 


(not from the main computer manufacturer). The standard 
communications software of the host Operating system is used 
and the host is not alleviated of the communications tasks 


meet. 127. 
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Another possibility of connecting a FEP to an already 
existent system, without the need of writing special software 
for the main computer operating system, is to use the standard 
software associated with different peripherals, such as disc 
drivers /Ref. 12/7. The FEP then protects the main computer 
from many of the communication tasks, and leaves it with only 


those that are common to all peripherals. 
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7 Figure 2.9 - FEP - host connection 


In the configuration of Figure 2.9 the main computer 
handles the file system, using its large disc or drum store. 
The FEP has a small disc which can hold message queues and 
files for a few users so that editing can be carried out 
here. Transfers between the two processors can be mainly 


in large blocks thus protecting the host from all rapid in- 


Ceractions. 





In some configurations there is no small disc attached to 
the FEP in Figure 2.9. Then, the only FEP storage is main 
memory and it does not handle some of the message Control 
functions. The FEP in such cases is typically responsible 
for communications circuit control, line switching and ter- 
minal control, activities which sometimes are called net- 
work and device control /Ref. 36/. 

In the configuration of Figure 2.10, called a disk-coupled 
system, the FEP shares a disc with the main computer /Ref. 
477. The FEP handles most of the communication tasks and 
assembles complete messages on the disk for attention by the 
main computer. Only when it recognizes a command from the 
user that needs the service of the main computer, does it 
send that machine a task, which it does by a message stored 
On a special region of the disk. Conversely when the host 
processor wants to communicate, it notifies the front-end 
processor of the attributes of the output file that it has 
Stored on the shared disk. The information exchange is thus 
done in a block basis. 

Another organization, the direct coupled shared memory 
System, requires that both the FEP and central processors 
have essentially the same architecture and internal timing/ 
control characteristics /Ref. 26; 47/. iiaci Is eCOnrt guration, 
primary memory replaces the disk as the shared coupling but 
with information exchange interleaved on a word rather than 


Block basis. The general advantage of the direct coupled 
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Figure 2.10 - Disk-coupled system 


system over the shared disk system is the ability to pass 
data and programs without the need for intermediate disk 
Storage. This configuration represents the most efficient 
integration of the FEP and host but it is usually only 
appropriate when the host and FEP are designed specifically 
for this application /Ref. 47/7. 

The shared disk and direct coupled connections provide 
away, after the architecture has been established, to con- 
figure a system that is essentially a dual processor. 
Larger architectures, on the other hand, are initially de- 
Signed as multiprocessors with integrated input/output com- 
puters or FEP's /Ref. 477. 

FEP's are used to remove data communications functions 


from the host, which can more Spelewenrly perrormeits data 
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processing functions. In networks with multiple processors, 
for the same reason, the hosts are frequently interconnected 
by a communications subnetwork or subnet /Ref. 177. The 
subnet is then an assemblage of communication circuits, FEP's, 
Switches and communication processors. The Figure 2.11 shows 
schematically the subnet concept. Besides all the advantages 
of alleviating the hosts from the communications load, the 
subnet provides well-specified interfaces to facilitate 
interfacing multiple computers from a variety of vendors, 
which otherwise would be very difficult or even impossible 

to interconnect in a large network /Ref. 177. In environ- 
ments with a single mainframe or small, homogeneous, single- 


vendor, networks the subnet concept may not be employed. 


H. BROADCAST COMMUNICATIONS 

Most of the experience with computer networks has invol- 
ved point-to-point circuits characteristic of common-carrier 
Systems. Yet the broadcast multipoint feature of radio 


transmissions is of utility in many situations /Ref. 1 


BL /. 
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A geographically widespread distribution of users may 
preclude joint usage of such a point-to-point communications 
facility. Depending upon the particular locations of the 
users, it can be more economical to provide each with a 
separate transmission path to the computer without the 


Savings derived from the use of sharing techniques, such as 
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Figure 2.11 - Network with communications 
subnetwork 


multiplexing, concentration and switching. Furthermore, 


commercially available transmission capability is often ill- 
Suited for the data communications required in some remote 

computer access applications. Another requirement, mobility 
in the military environment, cannot be met loi jalelsliqns (cle) joven eh= 


mircuits. 
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Under these conditions, netted operations with multi- 
point radio links can provide alternative means of communi- 
cations. By appropriately choosing the operating frequency 
and multiple-access technique, multipoint links have the 
potential of simultaneously connecting many users and pro- 
viding a reliable broadband digital communications capa- 
bility. In general, the organization derived from the 
use of a multipoint broadcast facility is distinct from that 
of point-to-point communications /Ref. 1/. 

Multiple-access techniques are disciplines imposed on 
the utilization of a broadcast radio channel. Multiple 
access, which permits sharing of communications capacity, is 
distinguished from multiplexing in that the latter refers 
to joint usage by several baseband signals, whereas the 
former is concerned with joint usage by several radio fre- 
quency (RF) signals /Ref. 1/7. Time division multiple access, 
TDMA, and frequency division multiple access TDMA are the RF 
analogs of STDM and FDM multiplexing, respectively. In TDMA 
a fixed time slot is cyclically allocated for each user of 
the channel over the entire bandwidth. In FDMA a fixed fre- 
quency band is allocated for each user all the time. Thus, 
they are called static reservation schemes /Ref. l, 45/. 

Typically, a stationary satellite has a broadband re- 
peater that simply amplifies all received signals and re- 
transmits them in a broadcast mode back to earth. The propa- 


gation delay for a roundtrip transmission (up and down) to a 
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Patellite transponder that is in a synchronous orbit is 
approximately 0.25s /Ref. 45/. If several terminals use the 
satellite at the same time, multiple-access techniques must 
be employed to ensure that links are not disabled by mutual 
interference, that the receiving terminals can identify the 
desired signals, and that efficient usage is made of the 
overall capacity of the satellite. Besides TDMA and FDMA 
there are other ways to use a given satellite channel for 
data communications. TDMA and FDMA divide the satellite 
channel in subchannels which are permanently assigned to 
users (ground terminals); this can be very wasteful in a 
bursty user environment /Ref. 31/. 

As alternative, 'random' access to the full capacity of 
the channel in a packet-switching mode can be employed. 
Again, a packet is defined merely as an addressed package 
of data that has been prepared by one user for transmission 
to some other user in the system. The satellite is charac- 
terized as a high-capacity channel with a fixed propagation 
delay that is large compared to the package transmission 
time. The transmission scheme to be considered is one 
wherein a particular transmitter assembles its packet and 
then, according to some discipline, transmits it rapidly 
on the channel at full speed. Many users operating in 
this fashion multiplex their messages on a demand basis 
/Ref. 1, 317. The satellite acts simply as a relay accept- 


ing the packets and broadcasting them back to earth; this 
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broadcast transmission can be heard by every user in the 
System including the packet source, l.e., each user can 
hear his own transmission; this is called ‘perfect feedback' 
or ‘automatic acknowledgement' /Ref. 30, 31/7. If one pac- 
ket arrives at the satellite transponder when another pac- 
ket is being relayed, both packets are destroyed. 

One decentralized approach to the sharing of the sat- 
ellite channel is the 'pure ALOHA' system. The name "ALOHA' 
comes from the pioneer computer network, built by the Uni- 
versity of Hawaii, employing this scheme. Ina ‘pure 
ALOHA', users are allowed to transmit whenever they want. 
If, after receiving back their transmission, the packet is 
intact, they assume the transmission was successful. Other- 
wise, a ‘collision’ with another packet has occurred or the 
packet was contaminated by noise; in such cases, they re- 
transmit the packet. If all users retransmit immediately 
m@pon hearing a conflict, then a collision will happen again. 
Thus, some scheme has to be used to introduce a retrans-~ 
mission delay to spread these conflicting packets over time. 
A random number generator is used to spread the time of 
retransmission in Baas to avoid a second collision. 

Another uncoordinated approach for using the satellite 
Channel is the ‘slotted ALOHA', where time is divided into 
segments (slots) of the same duration (assuming packets of 
constant length). Users are allowed to transmit only at 
the beginning of a time-slot (time referenced to the sate- 


llite); then whenever a packet is ready, they will hold it 
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until the next time mark. The probability of interference 
is diminished, because collisions are restricted to a sin- 
gle slot duration. 

The maximum throughput of a slotted ALOHA system is 
limited to 0.37 of the capacity of the channel, which is 
twice as much as a pure ALOHA system /Ref. 31/7. However, 
in a bursty user environment with light traffic, the slot- 
ted ALOHA system yields the minimum average delay. TDMA 
and FDMA lead to a maximum channel utilization (throughput 
approaching the channel capacity) in heavy traffic; the 
disadvantage of these latter processes is the delay in low 
traffic situations, which is bigger than that of the ran- 
dom access techniques. | 

For ground broadcast radio nets the same techniques for 
multiple-access may be employed. Typically, FDMA is not 
used in ground systems because it would introduce stringent 
requirements for frequency stability and filtering at both 
user and computer terminals /Ref. 1/7. Other methods em- 
phasizing simplicity of terminal design are economically 
more attractive. 

A fundamental difference of ground radio in relation to 
satellite communications is that the propagation delay is 
negligible compared to the size of a packet. Exploring this 
attribute, two more packet switching methods for multiple- 
access can be employed. 


In the carrier sense multiple access mode, CSMA, the 
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terminals listen ('sense') the channel; if the carrier sig- 
nal is present then the channel is considered in use by 
another terminal and transmissions are postponed until the 
channel is sensed as being idle. This information does 

not exist in satellite channels, because when the packet 
reaches the earth, the satellite transponder has already 
transmitted it. There are rules for deciding when a ter- 
Minal user may transmit and what action he must take if 

his transmission collides with another transmission. CSMA 
disciplines can lead to channel utilizations which approach 
unity /Ref. 31/. This is a sharp contrast to the poor 
utilization of ALOHA channels. 

The CSMA approach works on the assumption that all ter- 
Minals are within line-of-sight of each other. This is not 
always the case; in general, terminals are within the 
range of a central station (computer center, gateway to a 
network, etc.), but out of range of each other or separated 
by some obstacle opaque to radio frequencies. This Draicdens 
terminal effect' leads to another packet switching channel 
Sharing method, the busy tone multiple access mode (BTMA), 
because the existence of hidden terminals degrades the per- 
formance of CSMA. The operation of BTMA rests on the 
assumption that the central station is, by definition, with- 
in range and line-of-sight of all terminals. With BTMA, the 
total available bandwidth is divided into the information 


Channel and the busy tone channel. Whenever the central 
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channel senses an incoming signal in the information channel, 
Gt transmits the BT signal on the BT channel. It 1s by 
sensing a carrier on the BT channel that terminals determine 
when the information channel is busy. The performance of 
BTMA is almost as good as that of CSMA. With the use of 
BTMA, the delay due to hidden terminals is quite small /Ref. 


B1/. 


im. MULTIPOINT LINES 

An important method of increasing line efficiency is to 
use multipoint or multidrop circuits, rather than point-to- 
point circuits. This method connects a number of geogra- 
phically separated devices by a single circuit that passes 
from device to device. 

The Figure 2.12 shows a number of multidrop lines connec- 
ted to a computer and the Figure 2.13 shows two modes of 
connection for multipoint lines. Two wire connections are 
more suitable for short physical transmission lines such as 
for example the twisted pair of the local loop of a tele- 
phone cable. Four wire connections are used in common 
Carrier trunk circuits, which are intrinsically two-way 
channels. 

Two terminals cannot transmit simultaneously in multi- 
drop lines. The line controller or modem at the master 
position maintains the discipline of the line in some fash- 
Zon that allows only one transmission at a time. This inter- 


mittent nature of the use of a multidrop line may be dis- 
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guised by the use of buffers at the terminal. The buffer 
transmits into the line at high speed and communicates with 
the terminal at the user's rate, holding data until the 


proper time for transmission /Ref. 12/. 


COMPUTER v 


TERMINAL 


Figure 2.12 - Multipoint lines 


A multipoint line is only possible when the times of use 
of terminals can be interleaved and the sum of the average 
data transmission rate of each terminal does not exceed the 
Capacity of the circuit. There are many instances in which 
these conditions are satisfied as, for example, with inter- 
active terminals /Ref. 127. In addition to the savings in 
line costs, multipoint lines save in number of modems, since 
Only one is necessary at the computer side, while with point- 


to-point lines the number of modems is equal to twice the 


number of terminals. 
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C- CONTROLLER T-TERMINAL 





‘Figure 2.13 - Two (top) and four wire (bottom) 
multipoint connections 
The advantage of multidrop lines may be offset by a re- 
duced reliability and difficult of maintenance, when com- 
pared to point-to-point circuits. A failure in a single 


modem may make the line unusable and may be difficult to 


locate /Ref. 12/7. 


J. VALUE-ADDED NETWORKS 


Value-added network carriers are private companies who 
lease services from common carriers to augment these basic 


services through the use of additional physical facilities 
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(such as intelligent switching processors) and to resell the 
"higher value" to the public /Ref. 10/7. Thus, value-added 
networks (VANs) are built on the already existing communica- 
tions facilities to provide a different kind of public net- 
work service. A VAN utilizes the existing common carrier 
network for data transmission while providing the additional 
services of interfacing the users' computers and terminals, 
routing messages to their destination while guaranteeing 
their integrity, and allowing small users to take advantage 
of the economies of scale inherent in large communications 
systems by pooling their demands for service. Thus, a 
customer need only connect to a VAN in order to gain access 
to other computers and terminals throughout the nation with 
no required communications development on his own part. 
Basically, VANs provide communication subnetwork services. 

VANS improve the quality of the basic communications 
service in a number of ways. Through adaptive multiplexing, 
high speed lines can be shared by large numbers of users 
and more efficient use can be made of the available band- 
width. Alternate routing permits the network to maintain 
continuity of operations in the event of selective internal 
component failures. Powerful error detection techniques 
provides an end to end error rate to users that is many 
orders of magnitude better than a basic communications ser- 
Vice could provide. 

The intelligent components in a VAN can be employed to 


Provide new services to customers not provided by other 
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types of communications services. Among these services are 
speed recognition and conversion, code conversion, and the 
imposition of a set of network standard interfaces /Ref. 12/. 
Speed and code conversion permit a wide variety of different 
terminals to communicate with each other. As an implementa- 
tion of the subnetwork concept, the VANs provide a way of 
linking different computers by using standard interfaces. 
Besides technical benefits, VANS are economically 
attractive for small users and users who have varying vol- 


mmes of traffic. 
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III. FUNDAMENTALS OF GRAPH THEORY 


fe. INTRODUCTION 

Graphs are used to model an imense variety of problems 
and physical systems. For many systems the modeling by 
graphs is quite natural. 

Many aspects of computer networks can be easily repre- 
sented and mathematically studied by the use of graphs. 
One of such aspects, for example, is the structural model 
of the network. 

A graph may be considered as a set of points called 
nodes or vertices connected by lines called links or bran- 
ches. 

With each link and node of a graph, we can associate 
a number of parameters that represent the natural limita- 
tions and capabilities of the links and nodes. Those im- 
portant parameters that are incorporated into the model are 
Called weights. Weights are used in graphs to put non- 
structural information into the model. 

A graph can be defined as an abstract mathematical 
System. Diagrammatical representations of graphs provide 
motivation for the terminology and also help to develop 
some intuitive feelings. Figure 3.1 gives a diagrammatical 


mepresentation of a graph. 
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—>— DIRECTED LINK 
———— UNDIRECTED LINK 
O NODE 


Figure 3.1 - Diagrammatical representation 
of a graph 
B. DIRECTED AND UNDIRECTED GRAPHS 

Some links in networks have the attribute of direction 
as, for example, when modeling a communication channel which 
allows transmission of information in one direction but does 
not in the reverse direction. Other links are two-way links 
allowing flow in both directions. Thus, it is necessary to 
distinguish between directed and undirected graphs. 

A directed graph G=(V,B) consists of a set V of elements 
Called nodes and a set B of ordered pairs of nodes called 
directed links /Ref. 22/7. It is assumed that both sets, V 
and B, are finite. | 
The kth node in V is denoted by Vier where k is any lower- 


Case letter or numeral. Thus, if v has n elements, it can 
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be denoted as Va (vy, Vor sees Waa) Diagrammatically the 
Pienents of V are represented by circles or dots as in 
Figure 3.1. A link of B is represented by (i,j) when that 
link is directed from node Vi to node Wa Diagrammatically 
the link (i,j) is represented by a line with an arrowhead 
pointing from v, to ae 

The node we is called the initial node of the link (1,3j) 
while the node V5 is called the terminal node. Link (i,j) 
ms Called incident out of its initial node Vv; and incident 
into its terminal node We OF simply 1nclident ate Mis and wae 

The nodes V5 and v5 joined by the link (1,3) are called 
adjacent. 

The definition of graph does not permit more than one 
link from node Vi to node Vas while it is possible to have 
the links (i,j) and (j,1) simultaneously since they are 
Bastinct. 

If more than one link going from v; to v. is allowed 
but no more than S links from Wie to v. the resulting 
Structure is called a S-graph and the multiple links con- 
necting Vs to ee are designated by (1,3), (1,J)5, 

H,j), (xr <S). Figure 3.2 displays a 2-graph. 

Quite commonly the existence of a link (i,j) implies 
the existence of a link (i,j) in the reverse direction. 
In such a case the graph is said to be symmetric. When 


that happens, it is possible to replace the directed branches 


(1,3) and (j,i) by an undirected branch (i,3], which is 
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represented diagrammatically by a line without arrowhead, 


ee seen in Figure 3.3. 


BroGure 3.27 =—"AvZ=qrann 


(a) (2) 


Figure 3.3 - Replacement of a directed 
graph (a) by an undirected graph (b) 
An undirected graph G=(V,B) consists of a set of nodes 
Vand a set B of unordered pairs of these nodes called links. 
An undirected link is represented by (ao) while a directed 


link is represented by (i,j). 
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The concept of an undirected graph gives a concise and 
neat representation of symmetric directed graphs, for ex- 
ample when representing a computer network structure where 
two-way communication links (duplex or half-duplex channels) 


are used throughout the network. 


C. RELATIONS 

Graphs can model not only the physical structure of 
systems but any abstract concept that can be formalized as 
a relation. 

A relation Rona set S is any subset of the cartesian 
product SxS, or equivalently any set of ordered pairs (x,y) 
{ where xeS and yes. 

The system that can be modeled by the relation Rona 
set S can be modeled by a graph G=(V,B) by making V=S and 
B=R. 

Thus any abstract concept can be easily represented by 


a graph provided that it be modeled as a relation. 


D. WEIGHTED GRAPHS 

A weighted graph is a graph in which numbers are asso- 
ciated with links or nodes. Any number of weights can be 
associated with each node or link. 

The set of weights associated with node Vy is denoted 
by {W, (k) }. The set of weights. associated with link (i,j) 
is denoted by SURES 35) Ue 


A weighted graph is symmetric if wy, (1,3) =w, (J ,1) for all 
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ij and k. A weighted graph is said to be pseudo-symmetric 


ef 
2 (Cal = Fw, (44) 
1EV LEV 
Metor all j and k. 


E. PATHS AND CYCLES 
Let G=(V,, B,) and Go=(Vo, Bo) be graphs. The graph 


| G,=(V3, B.) is called the union of Gy and G. re =V UV 


3 2 


and B.=B,UB. and can be written as G.=G,UG,. 
A graph G,=(V,,B,) 1s said to be a subgraph of the 


| graph G=(V,B), 1f VicV and B.<¢B. Figure 3.4 illustrates this 


1 
concept, where CNA Vor Vay val, Pele 2 mC 2a.) ae aay 
(4,1), (1,3)}) and its subgraphs showed are | 
Go=({v,,Vg,vyh, ((1,3), (3,4), (4,1)}) and 
Go=({v,,va}, {(1,3)}). 
The degree d(j) of a node We is the total number of links 
| that are incident at Wa For a directed graph the inward 
_ demidegree d (3) of a node ae is the total number of links 
that are incident into Wat dora, We is the terminal node of 
| those links. The outward demidegree d” (3) of a node % 
is the total number of links that have Wr as initial node. 
Clearly, 
SG) See Gi) acl). 
Ej) =S (3) = S a (3) = [BI 
V.eV 


V.eV V.EV 
a) J J 
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where |B| represents the number of elements of the set of 


links, i.e., the number of links of the graph. 


V3 Vy Vs 
Vv, V, V;, 
GRAPH G SUBGRAPH G, SUBGRAPH G, 


Figure 3.4 - Subgraphs of a graph 











Emselt=lO60p LS GL the form (1,1) and contributes to both 
the inward demidegree d (i) and the outward demidegree d* (i). 
Many authors don't consider self-loops in the treatment of 
f graphs. Many times this is a matter of definition of the 
relation R which corresponds to the set of links of the 
graph. By considering that relation antireflexive, self- 
loops are excluded from the graphs. A relation R ona set 
S is said to be antireflexive if for every xeS (x,x) ZR. 

A graph is said to be homogeneous of degree x if for all 
V,eV, Ga ) =X. 

A directed path is a subgraph of G, specified by the 
me avence of nodes and links Vs (1),15)V; (15,13) re 


il 9 


68 








Vio (4,4) ty such that in the subgraph, d (i 


a" (i,)=1, ad (i 


1 
+) ee. 
pat, d (1,)=0 shovel epg aii lil thy eo pease 
1, d(i,)=d" (i,)=1, i.e., all nodes and links are dis- 
manct. 


An undirected path is a subgraph of G, specified by the 


sequence of nodes and undirected links Vs Li, /ip| 
al 


Vv. E ae, E iy) %5 such that in the subgraph 
1,2 3 4,7, *} 4, 


d(i,)=d(i =1l and for all ty ders Kat, d(i5)=2. The 


.) 
definition of undirected graphs implies that [i,j] =[5,4].- 

A path is a subgraph of G such that when link directions 
are removed, it is an undirected path. 

A path is not necessarily a directed path when link 
directions are considered. Sometimes the sequence of nodes 


and links is represented simply by the sequence of nodes 


Vi Vi ---Vy 
2 k 


or by the sequency of links (1,715) (15,13)... (A,_], i). 
Many times these sequences are referred simply as paths, 
Bae type of path being clear from the particular graph. 

The graph G is said to be connected if for any pair of 
nodes there is a path in the graph. Otherwise the graph is 
disconnected. 

A graph G is strongly connected if for any pair of nodes 
V. and V5 there is a directed path from v; to v5 and con- 


ol, 


versely for Ve to V;- 


69 





A subgraph is maximal with respect to a property P if it 
loses P when more links or nodes are added to the graph. A 
maximal connected subgraph of G is called a component. The 
components of G define a unique partition of the nodes of G. 
When G is connected, it has only one component. 

A directed cycle is a subgraph of G specified by the 
sequence of nodes and links Mo See ar ear 


V. (i 


) such that d@ (i.)=d° (i.)=1 for all j=1,2...k. 
*k-1 J J 


ead ook 


In other words, a directed cycle is a directed path in which 
the first and last nodes are the same; it is a "closed" 
directed path. 
, An undirected cycle is a sequence.of nodes and links 
with the same characteristics of an undirected path, except 
- the first and last nodes are the same. 
A cycle is a sequence of nodes and links satisfying the 


requirements for a path, except that the first and last nodes 













are the same, or, in other words, a cycle is a subgraph of 
G such that when link directions are neglected it is an 
undirected cycle. 

A graph that does not contain a cycle is said to be 


meyclic. 


=. CUT-SETS, CUTS AND TREES 
In dealing with problems of flows in networks and relia- 
bility, it is necessary to know what links and nodes contri- 


bute to the transmission of the commodity from one region of 
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the network to another. The total capacity of those links 
are to be considered in flow problems while in reliability 
problems the probability of failure of links and nodes must 
be additionally considered /Ref. 12, 22/. 

A set S is said to be minimal in relation to some pro- 
perty P if no proper subset of S has property P. 

An undirected link cut-set of an undirected connected 
graph is a Minimal set of links the removal of which yields 
a graph with two or more components. Figure 3.5 shows an 
example of cut-set. The removal of eae [apse [1,3], 
[2,4] ) from the graph is 3.5 a) CSTE Pie Graph. tn 3.5 
b) with two components. No proper subset of this set yields 


mea Gisconnected graph. 


Vi Vy Vi vy 
4% yw Up 
} 
7) V5 i) _ 
a) 4) 


Figure 3.5 - Example of cut-set 


When it is necessary to distinguish among the nodes that 
are separated by the cut-set, it is specified that an un- 


directed cut-set of an undirected connected graph is an i-7j 


oat 













cut-set when the nodes va and Ve are in different components 
meter the cut-set is removed from the graph. Thus, in the 
example of Figure 3.5 the cut-set ({2,4], papel ieee). 
[2,4]) is an S-4, S-t, S-3, 1-4, 1-t, 1-3, 2-4, 2-t and 2-3 
Sut-set. 

A set of links of a directed graph is a directed link 
cut-set if its removal from the graph breaks all directed 
paths from at least one node of the graph and no proper 
sub-set breaks all directed paths between the same vertices. 
Figure 3.6 gives an example of directed link cut-set; the 


met {((1,4),(1,3),(2,3)} is a directed branch cut-set since 


Vg Ue 
My, Me 
Figure 3.6 - Example of cut-set 


its removal destroys all directed s-t paths and no proper 
Sub-set does so. 
Again, to specify the nodes which are affected by the 


Cut-set removal, it is said that the cut-set is a directed 


ms-t link cut-set if it is a minimal set of links of the 
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directed graph whose removal from G breaks all directed s-t 
paths. 

The reliability of a network is not only affected by the 
removal of links; node removals must also be considered. 

When a node Vie is removed from a graph, all links inci- 
dent at v, are also removed from the graph. An undirected 
node cut-set of an undirected connected graph G is a minimal 

_ set of nodes whose removal from G separates the graph into 

two or more components. In the example of Figure 3.5 
{v,1Vo} and {v3,V4} are both undirected node cut-sets since 


both break all paths between Ve and v in this case it is 


¢? 
said to be an undirected s-t node cut-set. The removal of 
. an undirected s-t node cut-set from a graph breaks all un- 


directed s-t paths. In Figure 3.7 the only undirected node 










cut-set is {vj}. 


1 
{Dr 
% O—___0—_—_———o %& 


Figure 3.7 - Example of cut-set 


A set of nodes of a directed graph G is a directed node 
cut-set if its removal from G destroys all directed paths 
from at least one of the remaining to at least one other 
remaining node and no proper subset breaks all directed paths 
between these vertices. A directed s-t node cut-set is a 


Minimal set of nodes (not containing V, OL V,) whose removal 
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from G breaks all directed paths from node is to mode Vp: 


For a directed graph G, an s-t mixed cut-set is a minimal 
set of links and nodes, other than Ws and Vi: whose removal 
from G breaks all directed s-t paths. 

For an undirected graph an s-t mixed cut-set is a 
Minimal set of links and nodes not including se and Vie that, 
when removed, disconnects se from Vi: 

Closely related to the concept of a link cut set is the 
concept of a cut. Loosely speaking a cut is a set of links 


that connects a set of nodes to the remaining nodes. This 


is illustrated in Figure 3.8 for an undirected graph. 





Raginews 10 6— soo Cue 


More precisely let G=(V,B) be a directed graph. If 





Bev and QcV (P and Q not necessarily disjoint), let (P,Q) 
Benote the set of all links which are incident out of an 


meement of P and incident intwo an element of Q, i.e., 
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(P,Q)={(i,j)eB|v,eP and v.eQh. Given V,, a subset of nodes 


of G, let V, denote the set complement of V, in V. For any 


L I 


V,cV, the set of branches (V,,V,) is a cut. For example 


| 1 

p in Figure 3.6 if V,={v_} then ({vi}, {V4 1V91V3/V4/V4))= 
{(s,l),(s,2)} is a cut, while if Vi Ftv Vy 1Vo} then the set 
of links ({viiVy Vat, tvarVy/V,})={ (1,4), Cera) K2po) 4d 2s 


another cut. Any of these cuts can be called s-t cuts 


since v.ev, and v Sven eine general che motation (xX, xX) 


mel Se. 


where XeV represents any cut in which v.eX and Vex. The 
generalization for undirected graphs is simple: the set 
of branches [P,a| between two sets of nodes is defined as 

| [P,o]=( [i,j] Bl v,eP and V4€Q) and [v,,7, | Sy a) “Cube 

The concept of cut 1S more general than that of directed 


cut-set as can be seen in the following theorem. 










| Theorem - Not every cut is a directed cut-set. Every 
directed s-t cut set is an s-t cut. 

The same is valid for undirected graphics: every cut-set 
is a cut; not every cut is a cut-set. 
| Let G be an weighted graph in which the weight w(i,}) 
» associated with the link (i,j) denotes the capacity of the 
link See The capacity of an arbitrary subset of links 
BjCB is defined as the sum of all the capacities of the 


—— 


links of B,. That is 


i 
C(B,) = = Cis 


(1,j) eB, 


where C(B,) denotes the capacity of Bi: 
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ime the Gapacity Of each link is unity, then C(B,)=(B, | 


is the number of branches in B- 
| When an undirected link cut-set is removed, an undirected 
| connected graph becomes disconnected, but if a proper subset 
of that cut-set is removed the Seah remains connected. 
Different cut-sets have different numbers of links so the 
minimum number of links which can disconnect the graph when 
removed is not readily available, or the maximum number of 
links which don't disconnect the graph when removed is not 
available. Related to this last idea is the concept of a 
tree of a graph. 
| A tree T=(V,U) of a connected graph G=(V,B) is a connec- 
ted subgraph of G which contains all nodes of G but no 


cycles /Ref. 227. Figure 3.9 illustrates one possible tree 








for the connected graph of 3.9 a). 


Vy VU: 


5 Vu Ve av 
a.) £,) 
Figure 3.9 —- Tree of a connected graph 
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A tree T=(V,U) of graph G=(V,B) with n nodes has the 
following properties: 
| Meo U contains n-l Links. 
3 2. T=(V,U) is connected but becomes disconnected if any 
link is removed. 
3. Every pair of vertices Vs and Wie is joined by one 
and only one path. 
4. Any subgraph of G which contains T as a proper sub- 
Graph, has at least Bre cycle. 
5. Every undirected branch cut-set has at least one 


link in common with every tree. 


G. THE MAX-FLOW MIN-CUT THEOREM 


Let uS suppose it is necessary to transmit a commodity 





from a node oe to a node Vy through a network which contains 
these nodes. We wish to determine the commodity flow which 
the network can accomodate. The max flow min-cut theorem 
allows us to solve this problem /Ref. 1, 12, 22/. 

Theorem - The maximum flow from a node sa to node a Wins 
is equal to the capacity of the s-t cut which has the 
Minimum capacity among all s-t cuts. 

Mmtuitively, if we and V, are Migadurrebhene Groups. Of 
nodes, then all flow from NG to Ve must pass through the 
Cut that connects them. Therefore, the maximum flow from 
V, to v, cannot exceed the capacity of the cut; in particu- 


dar it cannot exceed the capacity of the cut of smallest 


Capacity. Figure 2.10 illustrates the application of the 


uy 








| 
j 
| 


ji 





max-flow min-cut theorem to an undirected connected graph. 
The maximum flow from Ve to vy is equal to two, the capacity 


of the s-t cut with smallest capacity. 
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@) UNDIRECTED CONNECTED GRAPH 





d) S-T CUT WITH CAPACITY 4 L) S-T CUT WITH CAPACITY 3 





Pigure 3.10- Illustration to the max-flow min-cut theorem 
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IV. COMPUTER NETWORK RELIABILITY 


| a INTRODUCTION 

: A factor of major importance to the successful operation 
of computer networks is their reliability. This factor be- 
ros increasingly significant as networks grow in size and 
their users become more dependent on their proper operation. 
Accordingly, one of the most important objectives in design- 
ing a network is to guarantee that it will function effec- 
tively even after some of its elements fail. Network relia- 
bility is strongly dependent on the topological layout of 
the communication links in addition to the reliability of 

| the individual computer systems and communication facili- 


ties /Ref. 12, 22, 527. 















The computer-communication network model that is gen- 
erally used for reliability analysis is an undirected graph. 
It has the advantage of simplifying the analysis while it 
‘ correctly represents most of the networks. 

Failures may occur in both nodes and links of a net- 

work. Careful design and duplication of equipment can 
Significantly reduce failure in the nodes at the expense 

of increased cost. Failures in links are not as controllable. 
To overcome link failures, networks can be designed with dup- 
lication of links, which may be very expensive, or may in- 
Corporate alternate routes between any pair of nodes, which 


may be more cost-effective. 
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The first step in studying the reliability of a system 
is to formulate precise criteria for its failure. These 
criteria will depend on the network function and its purpose. 
Reliability analysis of networks in concerned with the de- 
pendence of the network reliability on the reliability of its 
nodes and links. ene peeietes of a node or of a link may 
be easily defined as the probability of failure in time. 
The reliability of a network is much more difficult to define. 
Measures of the reliability of a network may be for example 
/Ref. 23/7: a) the number of elements that must be removed 
before the network becomes disconnected, i.e., before one 
pair of nodes in the network cannot communicate; b) the pro- 
bability that the network become disconnected; ¢c) and the 
expected number of node pairs that can communicate through 
the network. Another class of measures arises when the 
importance of the nodes in the network is not the same; some 
nodes may be more important than others due to nodal functions 


er resources. 


B. THE FORD-FULKERSON ALGORITHM 


The Ford-Fulkerson algorithm is used for calculations of 
flow in the one commodity case. Given a network, such as 
the one in Figure 4.l-a), the algorithm can calculate the 
Maximum flow that is possible from one node to another. The 
application of the algorithm to reliability calculations is 
for counting the minimum number of links in the smallest cut 


between a pair of nodes. 
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Figure 4.1 - The Ford-Fulkerson algorithm 
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The algorithm can be well illustrated by means of an 
example /Ref. 12/7. Consider the weighted undirected graph 
of Figure 4.la) where the weights represent the capacity of 
the links. In part b) a simple flow pattern is shown, one 

f which clearly does not exceed the capacity of any link and 
carries 7 units of flow from v_ to v,. The augmentation of 


S ie 
this flow is shown in part d) in which a path from Ve to ve 
has been chosen and an extra flow of 2 units added along 
that path. The result is a new flow pattern in e) which has 
Nine units passing from Vie to Vy. The ida tEVeuLey- OL” tne 
algorithm lies in finding the path along which the extra 
_ flow can be inserted. This part of the algorithm is based 


on labeling the nodes, and is shown in part c) of the figure. 


The labeling process starts at Vs Cache aI rection 1S 













tried.in turn. Direction 1-2 would allow 2 more units of 


Elow out of Vv, SO iheeiSed POSSiIbDle Stare tforvthe saeh. “le 


memotce this step, a label (1,2) is attached at node V5 with 
two items of information: the node Vy from which the flow 
from - might be received and the extra flow 2. The next 


direction to be tried is 1-4, where an extra flow of four 
units out of v; is possible and this is shown on label (1,4) 
at node 4. The third direction towards node 3 allows no 
extra flow and no label is attached to node 3 at this stage. 
Now node v is left out and it is marked as 'scanned' so that 
the labeling algorithm will not come back to it. 


To proceed with labeling, labeled node V5 is examined 
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meee che two directions of exit from 1t explored. The dir- 
ection of node Vy is of no interest because that node is 
already labeled. A flow of three of 3 units towards node 


v- is possible, which results in labeling node Ves With (2,2) 


3) 
to denote the flow of two which could be received via node 

V5 from Ve: At this stage, node Vo has been 'scanned' and 

is out of the running. At the next stage node Ve is chosen 
to be examined. It is important to note that if node v5 
were chosen a different result would have been obtained. 

But the object is only to find a path from v, tov, to augment 
the flow and where there are alternatives either one will do. 


The next node to be labeled is Vzz then finally v which 


6! 
is the terminal node. If there is a path for the flow to 

_ the terminal it will be found, though it may not be the 
most direct path. 

To complete the path-finding phase the path is retraced 
from Vy backwards. The possible flow, which started out as 
2, might have been reduced at any stage due to limitations 
On the capacity of the links. The final flow value of 2 
is shown on the label (3,2) at vy and this is the possible 
amount of the augmentation of this stage. The node refer- 
ences on the label allow the path to be retraced from v, as 
Vi-V3-Ve-Vo-V.. Then this flow is added to the initial flow 


Pattern as shown in 4.1d and the new flow is 9 as shown in 


4.le. 
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Now the whole labeling and augmentation process is re- 
peated. In Figure 4.2 the next flow of 11 was found and then 


the maximum of 12 was reached. The approach to the maximum 





FLOW 4 FLOW [1 
AUGMENTATION 1 





7) 
| 
R 
4, 
Uy 
FLOW 12 S-T CUT SET 
MAXIMUM 
Figure 4.2 - The maximum s-t flow 


and cut-set 





would be different if different augmentation paths were 
used, but the resultant maximum flow would be the same. At 


the final stage the process could not continue because in 
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the labeling process the tree extending from ee could not 
reach Vy and this marks the end of the maximization of the 


flow. Two nodes have been labeled. Together with node Vy 
they form the only part of the network accessible to an 
augmenting flow. From this set of nodes Vir Vo and V4 all 
the outgoing links (1,3), (2,5), and (4,6) are saturated. 
If follows that the set in question divides the network and 
S$ minimal; thus it is a link cut-set. It is also an s-t 
cut-set (and consequently an s-t cut) since it breaks all 
paths between ve and Vy: Then the augmentation process 


Stops because Me and v, are separated by a cut of saturated 


128 
links. 
The Ford-Fulkerson algorithm determines the maximum 


flow that is possible between Ve and v, and shows one 


ce 
pattern of flow which achieves this maximum. It is not 
always a unique pattern. The algorithm also determines an 


Ss-t cut which carries the full flow capacity in all links 


of the cut when the s-t flow iS maximum. 


C. PROBABILITY OF DISCONNECTION BY LINK FAILURES 

Given the network of Figure 4.la), let p be the proba- 
bility of failure of a link. Assuming that the failures 
of links are independent, the probability that a set of 
m links be out of service is a8 lao) Since the network 
has 9 links. To know the probability that ve and Vv, are not 


connected, it is enough to sum expressions like that for all 
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failure patterns that disconnect Ve from Vi- This would 
involve the examination of 2B We subsets of the set of 
nine links. 

En eee aT if a network has n links then the number of 
sets which have i links is noi: Out of this number only 
A. sets disconnect Vea pap ake) in| Mie if removed from the network. 
It is important to note that A. includes not only the s-t 
cut-sets but all those which have the s-t cut-sets as a 


sub-set. If P(s,t) represents the probability of dis- 


connection of ve from Vv, due to link failures: 
Z at n-l 
BMsiiee= =. Alp (1p) 
peal 


Tf the probability of any disconnection is needed, then 
A. must account for the number of sets with i links which 
cause any disconnection, instead of only those which dis- 
connect — from Vi: 

In real computer networks p is very small /Ref. 12, 23, 
527. Concentrating on very reliable networks, the calcu- 
lation of an approximate result for the expression above is 
Simplified because the first non-zero term is dominant. If 
the number of elements of the smallest s-t cut is denoted 
by @(s,t), the the first non-zero term in the equation has 
a=6(s,t). 

For small networks @(s,t) can be derived easily by in- 


Spection, but this is not the case for large and heavily 
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connected networks. The Ford-Fulkerson algorithm can be 
used for this calculation. It is necessary then to assign 
the capacity 1 to all links in the network and derive the 
Maximum flow from Vie to Vy- This value is equal to the 
number of links in the smallest s-t cut by the max-flow 


Min-cut theorem. 


The probability of Vig and Ve being disconnected by link 


failures is proportional to po (srt) 


2 pis small. Dn tais 
case the smallest value of 98(s,t) among all pairs of nodes, 
denoted by 98, is a measure of the vulnerability of the 
network to link failures. This quantity 9 is the smallest 
number of links that have to be removed to cause some dis- 


connection in:the network. 


It can be proved that the quantity 9(s,t) is the maxi- 








mum number of link disjoint paths between ve and v,- Two 
paths are link disjoint if they have no link in common, 


though they could have common nodes. 


D. NODE FAILURES 

Node failures are analyzed ina slightly different way 
than that of link failures and it is convenient to study 
ene effect of mixed failures, i.e., thogm@involving links 
and nodes. The main concepts are those of node cut-set 
and mixed cut-set. 

The number of elements in the smallest s-t node cut-set 
Yepresents the minimum number of nodes that must be removed 


from the network to break all s-t paths. This number is 
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denoted by v(s,t). It is possible to prove that /Ref. 12, 
o2/ 
WS 518) Sears) 2 
The node cut-sets and the related connectivity measures 
v(s,t) have the disadvantage of being unable to handle nodes 
that are adjacent, because v(s,t) is not defined. The mea- 
Ac of vulnerability of the whole network to node dropouts 
is the number v, the least of all v(s,t) in the network, 
which is the minimum number of nodes to be removed to make 
the network disconnected. When the network is fully con- 
nected no v(s,t) is defined but, by convention n=N-l for a 
network with N nodes /Ref. 12, 52/7. 
In the same way that 9@(s,t) and v(s,t) were defined, it 


1s possible to define u(s,t) for mixed cut-sets. It is the 














Minimum number of links and nodes which disconnects we and 
Vis when removed from the network. This number is more use- 
' ful than v(s,t) because it is defined for any pair of nodes 
and it can be proved that /Ref. 12, 22/7 

v(s,t)=u(s,t) 
wherever v(s,t) is defined. 

In the same way, u is the minimum of the numbers u(s,t) 
of all node pairs in the network. Again it can be proved 
that u=v 
including the case of fully connected network of N nodes 
where u=v=N-1. 


Analagous to @(s,t) and link disjoint paths, u(s,t) is the 
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Minimum number of node disjoint paths between Vie and Vie 
| Two paths with ie and Vv, as extreme nodes are called node 
disjoint if they do not have nodes in common other than v 
and Ve Of course, since two paths that are node disjoint 
are also link disjoint, the number of node disjoint paths 
is included in the number of link disjoint paths 

u(s,t)<6(s,t) 

The numbers u(s,t) are the best way of specifying 
the connectivity property for reliability analysis purposes. 
If only 9(s,t) is specified there is the possibility that a 
node or mixed failure could break the s-t paths. It is 
important to note that, because u(s,t)<0(s,t), node failures 
are more likely to disconnect non-adjacent nodes, if the 
probability of failure in nodes is of the same order as 
links: however, in networks this is not the case /Ref. 127 


Thus, u(s,t) should be specified for all node pairs or the 


value of u should be established. 


E. KLEITMAN'S METHOD 

To verify that u=x in a network involves the verification 
Maat u(s,t)>x for all node pairs. Using the Ford-Fulkerson 
algorithm and assigning unity capacity to each link and 
node other than the source and terminal, it is possible to 
derive u(s,t) for every pair of nodes. But this requires 
a maximization procedure for every s-t pair and would not be 
practical for large networks. The method due to Kleitman 


does this in an economical fashion /Ref. 1, 12/7. 
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Recall that u(s,t) is equal to the number of node dis- 


joint paths between Vie and Vy: First choose any node V4 and 


Meeaty that u(l,t)>x for every other node v that is, there 


t! 


are at least x node disjoint paths from v, to any other node. 


a 


' If successful delete Val and all links incident at it. Then 


choose another node v5 and Vierri vy thas (2, &) XI ror any 


other node Vi: If successful, repeat this procedure for a 
thired node V3 ange ciemeondilcion u(s,6)>x>2o0 Since x 
is usually a small integer the process will soon stop. To 


test that u(x,t)>0O it is only necessary to verify that the 
graph is connected. 
To appreciate the economy of the method, suppose it is 


necessary to analyze a 1000-node network to verify that 






u>3. This network has 500,000 s-t pairs and, without the 
method, would require 500,000 flow maximization calculations 
to do the job. Using the method would require 999 tests for 
u>3, 998 tests for u>2, 997 tests for u>l and no test for 
u>0. A total of 2994 flow maximization calculations would be 


performed, smaller by a factor of more than 100. 
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Vee LOPOLOGY 


me SYSTEM CHARACTERISTICS 

Many characteristics of computer networks are determined 
or influenced by their topology. Some qualitative attributes 
ment be inferred directly from the topology of the network 
independently of the particular implementation /Ref. Te 

For the purpose of comparison among the various types 
of networks some attributes will be defined as in the fol- 
lowing. 

Modularity, the ability to make incremental changes in 
system capability, can be viewed in two ways. First, the 
cost of adding an element to the system that will be called 
cost~modularity and second, the degree to which the place 
and function of the incremental element are restricted, will 
be called place-modularity. For instance, if for adding one 
more processor to the system, the incremental cost is that 
of the processor, the network exhibits a very good cost- 
modularity; but, if besides the acquisition of the processor 
it is necessary to install a communication link to each of 
the processors of the network, the system has a poor cost- 
modularity characteristic. There may be places in the net- 
work where it is easy to increase a specific performance 
Characteristic by the addition of physical resources (some 
Piece of equipment) and other places where it may be diffi- 
Cult or even impossible; in the former case the system has 


g00d place~modularity; this is not true for the later case. 


. 
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Connection flexibility measures the degree of freedom in 
adding an element (piece of hardware) to the network. In 
some systems there is no choice of how to connect a new ele- 
ment, while in others there may be alternatives with differ- 

Meent costs. 

One important aspect of systems is the ability for grace- 
ful degradation. In some systems the failure of one element 
May halt the operation, while in others degraded modes of 
Operation are possible. The failure-effect is a measure of 
the consequences of a fault in the network. The cost of 
graceful degradation is the cost of alternative methods for 
masking the faults to allow operation in a degraded mode. 


The cost of graceful degradation is low in systems where 












there is minimal spare hardware and no explicit reconfigura- 
tion is required and is very high where duplication of ele- 
ments is necessary and complex reconfiguration schemes have 
to be used. Of course, with minimal spare hardware avail- 
able, there is little capability for graceful degradation. 

In many systems there are inherent limitations in per- 
formance due to either the sharing of resources or non- 
uniformity of communications within the network. This pro- 
Bblem is referred to as one of bottlenecks. 

The architecture of a system is a major factor in deter- 
mining the number and nature of the decisions that must be 
Made to perform communications within the system. Logical 


” 


complexity is a measure of the intricacy imposed by the 
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type of network on the decisions that have to be made by 
) source, destination or switching elements during the communi- 


cations process. 


oe CLASSIFICATION 
Several schemes exist for classifying computer-communi- 
cation networks. Probably the simplest one divides them 
into centralized and distributed systems, according to the 
Ment to which processing power and data base management 


are distributed among the collection of host computers 












/Ref. 41/. 

Centralized networks provide a single host computer to 
service all users. In distributed networks, a variety of 
host computers may be accessed by network users. In simple 
words, centralized networks have one host computer, while 
distributed networks have two or more host computers. However, 
the presence of two or more host computers does not necessar- 


ily mean that the network is distributed. 


C. CENTRALIZED NETWORKS 

The most common architectures for centralized systems 
are the star and the tree shown in Figure 5.la) and 5.1b). 
Less commonly used is the loop configuration of Figure See Ko) 
Beef. 417. 

The star and tree configurations are both trees in the 
Sense of graph theory and thus have the same attributes 


eee th slight differences in the rank of some. The logical 
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Figure 5.1 - Configurations for centralized 
networks 
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Organization is the same for both. The use of concentrators 
and multiplexers is an implementation decision related to 
the economy in communication costs in the network and in- 
fluenced: by the geographical distribution of terminals. Re- 
mote access networks and time-sharing systems are typically 
implemented in one of these topologies. 

The loop configuration consists of a series of uni-~ 
directional links (simplex channels) interconnecting sequen-~ 
tially arranged stations along a single closed path. Mes- 
Sages circulate in the loop going from the central processor 
to each station and conversely from the stations to the cen- 
tral processor. Each station may contain one or a cluster 
of terminals. When a message arrives the loop interface of 
the station examines its address and transmits it to its 
adjacent processor in the chain or retains it depending upon 
Whether the message is addressed to it. 

With respect to the central processor, centralized 
systems have common characteristics: 

1. poor cost-modularity because it is not possible to 
add another processor; this characteristic will depend on 
the internal organization of the processor itself; 

2. poor place-modularity because the only place where 
the central functions can be enhanced is the central site; 

3. very poor failure-effect-a failure in the central 
Site stops the whole system; 


4. high cost of graceful degradation-only duplication 
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of the processor provides graceful degradation; 

5. the central processor is prone to be the bottleneck 
of the system; 

6. the logical complexity of the communications is low 
because of the centralization of control. 

The star configuration, in relation to terminals, has 
Mae characteristics of: 

1. fair cost-modularity and good place-modularity up 
to the capacity of the central processor; a new terminal may 
be placed anywhere but it will require a direct line to the 
central site; where multipoint (multidrop) lines are used, 
the addition of one more terminal to that line may be diffi- 
cult depending on the control structure and the control equip- 
meme Of the line; in ponencasae a direct connection may be 
Simpler or cheaper; 

me no connection flexibility, since the connections 
must all be direct to the computer; 

3. low failure-effect, because a faulted terminal 
affects only the place it serves; 

4. low cost of graceful degradation-the only action the 
System has to take is isolate the failed branch. 

Also the failure-effect of communication links is not 
Severe when it connects only one terminal to the computer, 
yet it is a bit worse in the case of multipoint lines. On 


the other hand, the cost of graceful degradation is moderate 


to high, because in some cases dialed telephonic lines may 
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be substituted for the failed private lines, while in other 
3 cases only duplication provides better degraded modes. 

The tree configuration has the same characteristics that 
the star with respect to terminals with two slight improve- 
ments: 

1. the cost-modularity is a little better, because it 
is possible to connect the terminal to a nearby concentra- 
mer Or multiplexer with short lines; 

2. the connection flexibility is a little better be- 
cause a new terminal may be connected to a concentrator or 
multiplexer or directly to the central computer. 

4 The drawback of the tree configuration in relation to 

' the star is its poor failure-effect and high cost of grace- 
ful degradation in relation to concentrators or multiplexers 
and high capacity links. A failure in one high capacity 
line, concentrator or multiplexer may disable a significant 
number of terminals. To improve graceful degradation dup- 
lication of concentrators or multiplexers is required, while 
for high capacity lines a dialed telephonic line may be used. 
If this is not possible, private lines can be duplicated. 

Centralized loop networks have, with respect to terminals 
The attributes of: 

i fair cost—-modullarity, fair connection flexibility 
and good place-modularity-a new terminal may be connected 
anywhere in the loop, may be connected to the nearest con- 


Centrator or may be inserted in the loop; in the last case 
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a cost is imposed in the form of the addition of one communi- 
cation path, a reroute of an existing path and a need for a 
loop interface; 

2. good failure-effect and low cost of graceful de- 
gradation-a failed terminal does not affect the rest of the 
system. The central processor isolates it from the network 
by not sending messages to it or retransmitting messages 
mom it. 

In loop systems the bandwidth of the communication links 
is an additional bottleneck. The characteristics of failure- 
effect and cost of graceful degradation are poor in loop 
architectures with respect to communication links and loop 
interfaces, where faults halt the operation of the system. 

To increase reliability, redundant communication paths may 

be used in the form of bidirectional communication links, 
1.e., another loop in the opposite direction, or a redundant 
loop in the same direction. Two loops in opposite directions 
complicate the design and have not been used. To overcome 


failures in the loop interfaces, some type of bypass must 


| be activated in the event of a fault in those interfaces. 


D. DISTRIBUTED NETWORKS 

The definition adopted for distributed computer net- 
works does not necessarily imply geographical dispersion 
Of host computers. The architectures discussed in the 
following sections may or may not have computers spread 


through a geographical region; that is an implementation 
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decision. Some topologies have better attributes than others 
for dispersion. 

In the following, only the characteristics of each archi- 
tecture derived from the particular type of interconnection 
will be discussed. The problems of connecting terminals 
are the same as for centralized systems because each host 
computer may have its own "local centralized network" of 
terminals. 

1. Loop 

Distributed loop architectures, seen in Figure 5.2, 
consists of a set of individual processors, each of which 
is connected to two neighbors by unidirectional links. The 
communication paths could be bidirectional, but the com- 
plexity caused by two-way traffic has prevented this imple- 
Mentation. Messages circulate in the loop going from source 
to destination, stopping at each processor, having its des- 
tination address examined until it reaches the destination. 

Loop systems have the characteristics of: 

a. Very good cost and place modularity. An addi- 
tional processor can be inserted anywhere in the loop with 
the addition of a single communication link sae the flow 
of messages is not significantly affected by its presence; 

b. The failure-effect is poor and the cost of 
Graceful degradation is high. A single failure in a path 
Or loop-processor interface interrupts the intercommunica- 


fion. To provide for graceful degradation, redundancy of 
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communication paths and a means of bypassing the loop-pro- 


cessor interfaces are required; 


NN 


PROCESSOR 


Pigunes..25— Og D 


c. The logical complexity is low. A processor re- 










ceives messages through a single port, transmits messages 
through a single port and must only relay messages not addres- 
weed to it and strip off those destined for it; 
) d. Low bandwidth in the communication paths may be 
m bottleneck. 

The bandwidth of the communication links together 
With the relay through processors causes delay in the trans- 
Mission of messages in the network. In some systems the 
Place-modularity is enhanced by the use of "soft" or "asso- 


_—_ ve" addressing; in this method messages are addressed 
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to processes instead of processors; each loop interface 


examines the address and sees if the addressed process is 


a 


. currently residing in its host computer. 
meeveep Wien Central Switch 

This architecture displayed in Figure 5.3, has 
characteristics in common with centralized systems with 
respect to the central switch and the characteristics of a 
loop with respect to the processors. 

In this topology messages are put in the loop by 
senders, removed for address translation by the central 
Switch and put back in the loop properly addressed to the 


recipient. 


me CENTRAL SWITCH 


~<Z_ processor 


Pgtse 5455 5— Loop with Central Switch 


OZ 











The loop with central switch has the attributes of: 

a. very good cost and place-modularity for pro- 
cessors; 

b. poor cost and place-modularity for the central 
switch; 

c. fair connection flexibility; 

d. very poor failure-effect and high cost of grace- 
ful degradation. The situation is worse than in the loop 
because of the central switch; 

e. in addition to the communication paths, the cen- 
tral switch may become a bottleneck; 

£. low logical complexity. 

3. Global Bus 
In this architecture, shown in Figure 5.4, a number 


of processors are connected to a common, or global bus. 


<2. PROCESSOR 


Figure 5.4 - Global Bus 
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Access to the bus is shared according to an alloca- 
tion scheme and a message is put on the bus by the source 
processor, which transmits it simultaneously to all others, 
and is received by the destination processor upon recogni- 
tion of the address. 

Bus architectures have the following characteris- 

metics: 

a. good cost and place-modularity with respect 
to the processors. Generally it is possible to connect a 
new processor anywhere to the bus with little or no effect 
on the other processors; 

b. very good failure-effect and very low cost of 
graceful degradation with respect to the processors- 
generally, failures in the processors do not affect the 
rest of the system and do not require any action by the 
System to reconfigure; 

c. catastrophic failure-effect and high cost 
pf pee nl daaeadation with respect to bus. In order to 
improve viability, replication of the bus is necessary; 

d. the bandwidth of the bus is a bottleneck in 
the network. 

4. Bus With Central Switch 

Piecwuomatenrtrecture, shown in Figure 5.5, the 
processors do not communicate directly as in the global bus. 
When a processor wishes to transmit a message, it must first 


acquire the bus, which is controlled by the central switch, 
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then transmit the message to the switch. From the switch 


the message is transmitted over the bus to the destination. 


sss ASO SECT. 
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Figure 5.5 - Bus with central switch 


The characteristics of this topology are similar to 
those of the global bus system: 

a. good cost and place-modularity with respect to 
the processor; 

b. poor cost and place-modularity for the bus and 
central switch; 

c. very good failure-effect and very low cost of 
graceful degradation with respect to the processors; 

d. catastrophic failure-effect and high cost of 
graceful degradation for the bus and central switch; 

e. in addition to the bus the central switch is 
a potential bottleneck; 

£. the logical complexity of this architecture 
@s less than in the global bus due to the control of the 


Switch over the bus. 
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5. Bus Window 
In this architecture, which is seen in Figure 5.6, 
processors are connected to buses controlled by switching 
elements connected to other buses. Switching is, thus, 
performed by more than one resource, and messages received 
from one bus may be retransmitted onto the same bus or onto 


another bus. 
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Figure 5.6 - Bus Window 


a. very good place-modularity and cost-modularity- 
processors and communication paths may be added where and 
when needed, and processors are only required to have one 
connection to the system; 

b. poor failure-effect and high cost of graceful 


degradation with respect to switches and buses. A failure 
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in a switch or in a bus can stop a significant part of the 
system and only replication can add graceful degradation; 

c. very good failure-effect and low cost of grace- 
ful degradation for processors; 

d. buses and switches are potential bottlenecks; 

e. the logical complexity increases as the system 
grows in number of buses and switches, requiring more levels 
of address translation; also the logical design must be 
done carefully because this system is subject to deadlock. 

6. Complete Interconnection 

The complete interconnection or fully connected net- 

work is the most conceptually simple network. Each processor 


] is connected to each other by a dedicated communication path. 





When a processor has to transmit a message to another one, 
it chooses the appropriate port from the alternatives avail- 
able. All processors must have the ability of receiving 
messages at multiple ports. 

This topology, shown in Figure 5.7, has the attri- 
butes: 

a. poor cost-modularity. The addition of a new 
processor requires connections to all processors already 
in the system; this requires the processor to have at least 
One port free to accept the new processor; 

b. good place-modularity; 


c. very good failure-effect and low cost of grace- 


ful degradation-since a failed processor does not affect 
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the communication paths between other processors, no recon- 
figuration action other than isolation of the failed pro- 
cessor is necessary; failures in the communication links 
are handled by routing messages through intermediary pro- 
cessors between source and destination; 

dad. no connection flexibility; 

e. there are no bottlenecks; 

f. relatively low logical complexity. Due to the 
need for switching messages after a failure occurs in a 
communication path, the design has to incorporate location- 
addressing capability for interprocess communication; de- 
pending on the level of graceful degradation needed, the 


logical complexity may increase substantially. 


Figure 5.7 - Complete interconnection 
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The distributed star architecture consists of a set 
of processors connected by means of dedicated bidirectional 
paths to a central switch, which accepts messages from the 
source and delivers them to the destination processor. The 
Switch also performs the function of isolating processes 
running on a particular processor, preventing them from 
having knowledge of the system and protecting them from each 
other. | 

Star systems, schematically shown in Figure 5.8, have 
most characteristics in common with the global bus archi- 
tecture, because they too share a central communication 
facility: 

a. good cost and place-modularity with respect to 
the processors and poor in relation to the central switch; 

b. good failure-effect for processors and poor for 
the switch; 

c. low cost of graceful degradation for processors 
and high for the switch; 

d. poor connection flexibility because an additional 
processor can only be connected to the switch; 

e. the central switch is the bottleneck of the 
system; 

£f. moderate logical complexity-the switch must keep 
track of the status of the system and store addressing 


Cables for translation of logical addresses. 
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Figure 5.8 - Star 
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Figure 5.9 —- Multiprocessor 
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8. Multiprocessor 

This architecture, in which two or more processors 
share a common-accessible memory, has not been considered 
a computer network. The common memory is used as a communi- 
cation path among the processors as well as a storage med- 
ium. Communications can be very fast because they can be 
performed by pointers to the messages that are interchanged. 

The multi-processor architecture has some character- 
istics of the star and global bus due to centralization of 
communications in one device, but the nature of memory as an 
inherent and subordinate part of a computer imposes differ- 
ences: 

a. good cost and place-modularity for processors 
and memory. It is possible to add processors and to in- 
crease memory size; the path structure by which processors 
access memory has a great influence on cost-modularity; if 
each processor has a private path to memory, the number of 
ports in memory limit the number of processors; if memory 
is accessed through a common bus, cost-modularity is very 
good, since processors can be easily connected by the bus; 

b. poor cost-modularity for memory bandwidth; 

c. memory bandwidth is a major bottleneck of the 
system; 

d. very good failure effect and low cost of grace- 
ful degradation for processors; 

e. poor failure-effect and high cost of graceful 
degradation for central-memory and memory bus; 
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f. moderate logical complexity-careful design is 
required nevertheless, to avoid deadlocks and race condi- 
tions. 

9. Regular Interconnection 

In this architecture every processor is connected to 
an equal number of other processors, providing identical 
neighbor relationships, or, in terms of a graph, every node 
has the same total degree. The loop is a special case of 
this topology. Messages are routed from source to destina- 
tion, with each intervening processor deciding which of its 
neighbors should relay a message. 

The characteristics of this design, one example of 
which is shown in Figure 5.10, are: 

a. bad modularity and failure characteristics be- 
cause of the requirement for regularity; 

b. logical complexity is moderate-it is increased, 
nevertheless, if reconfiguration after failure is to a irre- 
gular structure; 

Cc. no connection flexibility. 

There has been no practical implementation of this 


Structure, due to the characteristics above. 
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Figure 5.10 - Regular interconnection 


10. Irregular Interconnection 

This topology differs from the previous one in that 
there is no requirement for regular neighbor relationships. 
A processor may be connected to one or more other processors 
in the network. Most of the implemented distributed com- 
puter networks are in this class. Many of the system char- 
acteristics vary with the degree of irregularity of inter- 
connection. 

General characteristics of this topology, one case 


of which is seen in Figure 5.11, are: 
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a. extremely good place-modularity-processors and 
communication links are added when and where needed; 

b. very good cost-modularity~-additional processors 
are architecturally required to have only one path to the 
rest of the system; other principles of design, neverthe- 
less, may require at least two or three connections: this 
is not a requirement of the topology but, rather a relia- 
bility requirement; 

Cc. very good connection flexibility-any processor 
may be linked to any other processor in the network; 

d. good to very good failure-effect and low to 
very low cost of graceful degradation-the more irregular 
the Bani eooe are. the less is the effect of a failure 
(processor or communication link) and the easier are the 
reconfiguration actions after a fault, due to the existence 
of multiple paths (node disjoint) between any two processors; 

e. high to very high logical complexity-each 
Switch requires knowledge of the overall system status; 

£. bottlenecks are not a problem and when satura- 
tion takes place in the network, it can be easily alleviated 


due to the good place-modularity of the architecture. 
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Figure 5.11 - Irregular interconnection 


E. Geographic Dispersion of Distributed Topologies 

Not all distributed architectures are characterized 
by geographical dispersion. Communication line cost may 
limit dispersion. 

In this sense, good architectures for distributed 
networks are irregular interconnection, star, loop types 
and bus types. Perhaps the dominant topology in large 
computer networks is the irregular interconnection, which 
does not impose any pattern for interconnection and is de- 
Signed according to optimization principles, such as minimum 
cost, minimum average message delay, maximum throughput, 


Minimum communication cost, etc.; then the cost of high 
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logical complexity is largely compensated by the minimiza- 
tion of expensive communication lines. 

Multiprocessor systems require very large bandwidths 
for processors-memory communications and thus are generally 
confined to one room. This is the reason they are not commonly 
considered to be computer networks. 

Virtually all existent geographically dispersed, 
completely connected networks have small numbers of proces- 
sors, i.e., less than four processors /Ref. 2/. The communi- 
cation cost of a network with a large number of processors 
spread over a large area would be prohibitive and line utili- 
zation would be low. 

The star architecture can easily be spread over large 
areas. In many cases the star is the most economical way 
of implementing a distributed computer network because it 
intrinsically minimizes the number of communication lines 
and has low logical complexity. 

Loop systems have not been used for dispersion over 
large areas. Typically they have been implemented with 
transmission lines to integrate a set of minicomputers 
Perhaps the main factors preventing greater use of loops is 
their poor failure-effect in relation to communications 
links and the delay associated with the transmission of 
messages when the loop has many nodes. Also, for large 
distances, the natural solution is to employ leased common 


Barrier circuits; often such circuits are bidirectional and 
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one direction of transmission would be wasted in a uni- 
directional loop. 

Bus systems are used in conjunction with broadcast 
radio channels /Ref. ae Cie This 1S an inexpensive way of 
implementing a network in remote areas; in transcontinental 
networks, bus architectures used with satellite channels 
May achieve great economy in communication costs. When 
the bus is implemented on a point-to-point basis, the system 
is generally confined to one room because for short distances 
it 1S economical to implement a parallel bus. If a serial 
bus is employed, the system may be spread more by using 


mransmission lines. 
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VI. NETWORK INTEGRATION 


A. INTRODUCTION 

A computer network is not just a set of hardware and 
' communication circuits. The transformation of this set of 
resources into a computer network is performed by a set of 
rules and conventions which enables.the conversation among 
the network components and an orderly and efficient use of 
resources /Ref. 9/. Thus, this set of rules and conventions 
integrates and gives unity to the computer network. 

In any communication system and in particular in com- 
puter networks such a set of conventions and control pro- 
cedures iS necessary to allow efficient, smooth and correct 
transfer of information in the system. The main purpose of 
these rules and conventions are /Ref. 46/: 

j 1. to make the system convenient to use 

2. to prevent loss of data 

3. to detect message duplications 

t 4. for efficient and orderly use of resources (lines, 


| communication processors, etc.) 


5. for error detection and correction 

6. to detect system element failures 

7. for recovery from system failure 

8. to prevent and recover from traffic deadlocks 
9. to prevent congestion. 


The complexity of these functions varies from low in 


118 











centralized networks to very high in irregular distributed 
networks employing packet switching. Most functions are 
implemented in software; typically the low level functions 
such as link control are performed by hardware and the high 


level, complex function are performed by software. 


B. INTERFACE COMPONENTS 

An interface is an entity between two components of a 
system or two systems that serves to connect them. The 
interface may be a system, component device, or set of speci- 
fications /Ref. 10/. 

Computer communication interfaces are composed of both 
devices and specifications. There are many different ways 
to classify computer communication interfaces. One possible 
way is to divide the interfaces according to different 
functional levels: physical, electrical, logical or pro- 
cedural. 

The physical portion of the interface specifies the way 
fm which the two devices are actually connected together 
mechanically. This includes the number of wires and the 
dimensions of the physical connectors (generally a male and 
female connector are specified) in which the devices ter- 
Minate. 

The electrical portion of the interface specifies the 
voltage levels, frequency, impedance and other electrical 


attributes of the various leads. The basic capability pro- 
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vided by adherence to standards at this level is the trans- 
fer of bits of data across the interface. These bits may 
represent characters being transferred or higher level con- 
trol signals within the interface. 

The logical portion of the interface specifies how the 
bits of data are grouped into fields for the purposes of 
data transfer and signaling. The use of certain special 
characters for communications control is specified at this 
level as, for example, synchronization (SYN) and start-of- 
header (SOH). The logical specification plays the role of 
the language by means of which data is exchanged through 
the interface. 

If the logical level of the interface is viewed as 
specifying the syntax of the data flow across the inter- 
face, then the procedural specifications should be viewed 
aS providing the semantics. Specifications at this level 
determine the legal sequences of communication control 
characters, or the legal contents of various fields, or 
the valid commands and responses in controling data flow. 
The same basic set of control characters or fields may be 
used in a variety of different ways according to the pro- 


cedural specifications. 


=- PROTOCOLS 
The term protocol is generally used to refer to the 


logical and procedural aspects of an interface. Thus, a 
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protocol specification includes both syntax and semantics. 
Semantics also include information about legal use of re- 
sources-not only the legal commands, but who can issue 
them and when. 

A complex interface may contain several levels of pro- 
tocol. An appreciation of this aspect of protocols is 
quite important in designing and evaluating network inter- 
connections. 

Link control disciplines are in most cases capable of 
being implemented in hardware, although they could be im- 
plemented in software, and therefore be considered as a 
type of process-to-process protocol. 

Link control protocols are Siaine: bit-oriented or 
character-oriented. In both cases, there is a way to frame 
discrete units of information, a set of commands and a way 
to convey them, and an error detection mechanism. 

; In computer networks, most protocols are for the ex- 
Change of information between processes. In the case of a 
Simple terminal, the actions of the human operator are con- 
Sidered to constitute the process. In fact, when somebody 
logs on a computer system, a process is created to manage 
the terminal communications, thus allowing the device to be 
treated as any other process in the system. In this way, 
just one type of high-level communication exists in the 
system-the interprocess communication; the particularities 


of any device (not just terminals) are disguised by the soft- 
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ware system which handles the device. 

Many different processes may reside in the same physical 
device. For example, a host computer may have a communica- 
tions handler process, various operating systems processes, 
and many user (application level) processes. Messages 
entering the host through the same physical interface may 
be intended for any of these processes. In most cases there 
is a hierarchy of control for delivering messages to the 
proper recipient. The communications handler must examine 
all messages; the operating system must examine the messages 
intended for user processes. Thus, the basic structure for 
all messages must have some mechanism for identifying level 
and recipient. Frequently, information for recipients at 
multiple levels may be conveyed by the same physical message. 
One example is the method used in most packet switched net- 
works, in which messages are simply nested within messages, 
according the relative position of the recipient in the 
hierarchy. Each recipient "peels off" the part intended for 


him and passes the remainder to the next level recipient. 


D. CONTROL REQUIREMENTS 
There are a variety of control-related functions that 
must be performed by all computer networks. These functions 
include addressing, signaling, flow control and error control. 
In a communications system, some means of addressing is 


required whenever sender and receiver are not directly 
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connected. With a multi-channel terminal such as a host 
computer, which also contains different levels of addresses 
(such as operating system level, user program level), the 
question of addressing the proper recipient is non-trivial. 

Signaling refers to the means by which control infor- 
Mation is exchanged inside the network. The two primary 
classes of signaling techniques are in-band and out-of-band 
Signaling. In the former, control information is exchanged 
in the same way that data are exchanged, with some special 
identification marking it as control; in the latter, some 
other means is provided for exchanging control information, 
separately from data. Where nested protocols are used, 
control information sent at a different level of protocol 
May be considered as out-of-band signaling, even though all 
the data at all levels of protocol are transported by the 
Same physical means. 

Flow control refers to mechanisms for varying the data 
rate of data exchange between any two points in order to 
prevent overload. With probabilistic message generation 
and fixed capacity in network components (such as buffers 
and communication circuits), overload would be inevitable 
without such mechanisms to temporarily stop or slow down the 
rate of message arrivals. 

Sequencing and acknowledgements of messages can be con- 
Sidered as part of flow control or can be treated separately. 


The question of sequencing is generally treated as part of 
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the service to be rendered, and not as a mechanism that can 
be implemented in a variety of ways. Message acknowledge- 
ments, on the other hand, can be accomplished in a number 

of ways, and is frequently integrated WEE CLOW CONELOL DLO 
cedures. 

Just as message exchange can occur between parties at 
various levels in a hierarchy, so too can flow control be 
implemented for any of these levels. Of course, the actions 
of a flow control mechanism at lower levels in the hierarchy 
(at the communications handler, for example) would also be 
effective for all higher levels. 

Error control mechanisms also are implemented at various 
levels. While lower levels of an interface may adequately 
address the question of error detection and possibly even 
retransmission, somewhat higher levels may have to become 
involved with the detection of duplicate data occuring due 
to retransmissions after timeouts and to reinitialization 
after a catastrophic failure. Message acknowledgement may 
also be implemented as part of an error control scheme, and 
in fact, error and flow control can be based on the same 


mechanism. 


E. ROUTING 
In tree-like networks the problems of connecting (phy- 
Sically or logically) two points are straight-forward, 


Since there is only one possible path between any two nodes. 


124 


-=@ 





Irregular networks, on the other hand, present the greatest 
difficulty for finding a route between two nodes. 
Routing algorithms may be classified as /Ref. 25/: 
1. deterministic 
a. flooding 
pe Lixed 


Qe _yodsakie, gigs ial 


d. ideal observer 
2 seochastic 
ie random 


b. isolated 
c. distributed 
Routing algorithms are used in circuit, message, and 
packet switching networks. For data communications purposes 
the emphasis is on routing schemes for messages or packet- 
Switching networks. Several of the methods of routing were 
developed for circuit-switching in the telephone network. 
They were afterwards extended for message and packet- 
Switching networks. 
1. Deterministic Algorithms 
Deterministic routing algorithms derive routes 
according to a rule specified in advance. Each route pro- 
duces loop-free routing, so that messages can never become 
trapped in closed paths /Ref. ZO; 41/. | 
Flooding is perhaps the simplest of all routing al- 


gorithms. According to this algorithm, a node which re- 
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ceives a message immediately retransmits it over all connec- 
ted lines except the one from which the message was received. 
After the message has circulated through the network for a 
specified period, a message is returned to the node of ori- 
gin as confirmation that the flooding cycle has been com- 
pleted for that message. 

Flooding always routes messages with minimum delay 
and does not require large space for storage because it 
does not maintain current routing information or build de- 
lay Meee ar ine mechanisms. On the other hand, after a trans- 
ent period, a network employing flooding will quickly be- 
come congested, because of the excess of traffic in the net- 
work. 

Flooding has been suggested aS an initial “path 
finder" to derive routing and path delay statistics that 
other techniques require. However, efficiency considera- 

tions rule out flooding as a practical policy for network 
routing. 

Fixed routing, another deterministic algorithm, 
, assumes fixed topology and known traffic patterns. It re- 
duces optimal route selection to a multi-commodity flow 
with well-defined techniques for solution. This algorithm 
usually obtains appropriate routing from a directory in the 
memory of the node computer; the directory is fixed for any 
particular network configuration. A routing directory con- 


| tains the link address for sending a packet message from 
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a node to any location in the network. By searching the 
directory for a given destination, the node obtains a cross- 
reference to the corresponding link for transmitting the 
packet. 

Because of their inflexibility, fixed routing tech- 
niques do not present good graceful degradation. To improve 
this aspect, modified algorithms that include alternative 
fixed routes are required. 

Split traffic routing, sometimes called traffic 
befurcation, allows traffic to flow on more than one path 
between a given source and destination. If two different 
paths, Ry and Ro, are avallable, a packet at node S would 
be routed over R, with probability p and over R 


it Z 


bability (l-p). Similarly, traffic can be split over more 


with pro- 


than two routes with a different probability for each-the 
Sum of all probabilities being 1. This eieomt cli uses a 
directory that lists all the alternative routes, the pro- 
bability for each route, and a record of past choices that 
helps establish the current choice. Split traffic, when 
compared to fixed routing, maintains a better balance of 
traffic throughout the network, thus achieving smaller 
average message delays. Nevertheless, in terms of through- 
put and delay, split traffic always turns out to be some- 
what less than optimum. 

A fourth deterministic algorithm, the ideal observer 


routing technique, requires total knowledge of the network 
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in a continuous fashion. For each new packet entering the 
network, the node computes a route that minimizes the tra- 
vel time to the packet's destination. This computation is 
based upon complete present information about packets that 
previously entered the network and the routes that were 
computed for them. Because of inherent network delays, the 
ideal observer technique is only of theoretical interest. 
For example, when a packet reaches its destination, completes 
a message, and leaves the system, the destination node in- 
forms the source node of this fact, but the source node 
doesn't learn about it until after the advisory packet has 
worked its way back through the network. Thus the observer 
can't know what the network is like now, only what it was 
a few moments ago. Also the amount of information that has 
to circulate aa the network to implement this scheme is too 
great to be of practical use. Nevertheless the concept of 
"total knowledge" is useful in establishing an upper bound 
On network performance. 
2. Stochastic Algorithms 

Stochastic algorithms are probabilistic decision 
rules, as opposed to deterministic rules. They Sees routes 
in accordance with network topology, perhaps combined with 
Brstes of the state of the network. meee estimates are 
based on statistically derived delay information transferred 
from node to node between packets. Each node maintains a 


routing table for this delay information, and updates the 
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table whenever new information becomes available. The 
algorithms use the information in the table in much the 
‘Same way that the split-traffic and fixed-routing algorithms 
use their directories and the number of routes (number of 
table entries) may be greater than the number of links from 
the source node /Ref. 12, 25, 41/. 

Random routing algorithms assume that each node 
sends its received messages forward along a link chosen at 
random. The message eventually arrives at the destination 
after following what is sometimes called a "drunkard's walk". 
The algorithm can include a bias to guide the message roughly 
in the right direction, but should retain a substantial ran- 
dom element to cope with possible link or node failures. 
Although such algorithms are inefficient, they are surpris- 
ingly stable in networks having high probability of link or 
node failure. 

Isolated routing, using local delay Pe ernacaee 
assumes that traffic loads are approximately equal in both 
directions between any given source and destination pair. 

| This technique, also called "backward learning", uses as 
the estimate a weighted version of the time delay incurred 
by messages going in the reverse direction from the destin- 
ation node, considered as a source, to the node in question. 
The algorithm updates the previous delay estimate for trans- 
Mitting a message from the current node to another node 


through a specific link when a message from that node arrives 
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through this link carrying the time it wae traveled across 
the network. A simple linear recursive equation is used 
for delay estimate updating. This technique was found in 
practice to suffer from a "ping-pong" or looping effect, 
in which messages sometimes return to a node from which 
they were previously transmitted. In addition backward 
learning was found to adpat poorly to damaged networks. 
Another isolated routing technique, called the 
isolated shortest queue procedure, stems from the develop- 
ment of an adaptive routing method for military voice com- 
munications that would survive well in the event of nodes 
and links that have high failure probabilities (war zone). 
The procedure, sometimes called the "hot potato" method, 
requires intermediate nodes to retransmit a packet as 
quickly as possible after receiving it. Each node in the 
metwork, consulting a ranked list of lines leading to 
| neighboring nodes for every destination, directs packets 
to the highest rankding free line for a given destination, 
' or, if none are available, to the line having the shortest 
/ queue. Thus, the nodes handle the BAe (messages) like 
hot potatoes, getting rid of them as soon as possible. 
Distributed algorithms rely on exchange of observed 
delay information between nodes. This approach introduces 
an inordinate amount of measurement information into the 
network and is therefore impractical for large networks. 


One modified procedure uses a "minimum delay vector"; another 
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has an “area approach". 

A minimum delay vector for a particular node is the 
delay from that node to each of its adjacent nodes - rather 
than to all the other nodes, as in the unmodified algorithm. 
Each node exchanges this vector with each of its adjacent 
nodes; they update it and pass it on to their neighbors. As 
these vectors pass along the nodes, they eventually provide 
each node with a matrix of delays to all possible destina- 
tions. Exchanges and updates can be repeated periodically 
or irregularly. 

The area approach partitions the network into dis- 
joint areas. Within each area, every node exchanges infor- 
matiOn with every other node, but it exchanges similar in- 
formation with adjacent areas as though each were a single 
node. This approach can be extended to a hierarchy of 
routing clusters at many levels. The objective of the area 


approach is to reduce the amount of routing information that 


' €ach node must retain. 





An algorithm that combines stochastic and topological 
features uses a network routing center to which each node 
periodically sends updated traffic information. With this 
information, the network routing center regenerates routing 
tables, which remain fixed until the next update. This tech- 
nique has two disadvantages: like other centralized net- 
works, it has a single switching point (the NRC), and sin- 
g'.e paths for each source-destination pair constrain system 
behavior. 
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A hybrid algorithm with respect to centralization of 
routing divides routing decisions into two categories: those 
that affect the network only locally, implemented at the node 
level, and those with global effect, entrusted to the net- 
work routing center. This algorithm, called delta routing, 
appears to take advantage of the favorable aspects of both 
centralization and distribution of the routing function, but 
still has the weakness inherent in a central Senerol Facility 

3. Routing Performance Measures 

The performance of a routing algorithm must be con- 
Sidered in terms of four factors of fundamental importance 
for the network as a whole /Ref. 12, 25, 31/. 

The first factor is delay. The theoretical minimum 
for delay is determined by the network topology and the 
traffic levels of the moment. The routing program should 
come as close as possible to that minimum, particularly with 
respect to interactive traffic. 

Throughput is another factor to be evaluated. 
Throughput considerations parallel that of delay in the sense 
that there is a maximum throughput level for a given net- 
work topology and traffic mix. There is a tradeoff between 
delay and throughput. High throughput rates are most impor- 
Meee £0r bulk traffic. 

GeOceetseene Ehicd factor. The routing algorithm 
can affect the cost of network utilization by its demands 


for three resources: line bandwidth, node bandwidth, and node 
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storage. Routing over as few lines and nodes as possible 
and choosing under-utilized lines and nodes can keep band- 
width demands down; and keeping network queues short can 


reduce storage requirements. 


F. FLOW CONTROL ALGORITHMS 

Congestion can occur within a nework employing packet- 
or message-switching either globally or locally. It can be 
relieved by the use of a hierarchy of protocols that in- 
dicate which of several alternative actions is appropriate, 
on the basis of information from message and packet control 
fields. The hierarchy consists of host-to-host, source-to- 
destination, and node-to-node protocols, corresponding res- 
pectively to action at the message, packet, and link levels 

of a packet-switching network /Ref. 25/. 

Node-to-node or link protocols are local; host-to-host 
and source-to-destination are global, or end-to-end, since 
they control traffic into the network. Three flow control 

schemes among many utilize one or more of these protocols: 
: isarithmic, buffer storage allocation, and special route 
assignment. 

In an isarithmic network, the total number of packets 
is held constant by replacing outgoing data-carrying pac- 
kets with dummy packets. A dummy packet has a special 
identification block, is part of a one packet message, is 
always addressed by any node to its adjacent node in the 


path in which the dummy is inserted, and contains a 
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meaningless text block. New packets enter the network in 
place of the dummy packets. 

With buffer storage allocation the source node requests 
allocation of message reassembly space in the destination 
node before transmitting the message. The alternative, to 
go ahead and transmit the message, occasionally would find 
the destination node's receiving buffer full when the traffic 
was heavy, in which case the packets would not clear the 
last link in the path from source to destination and would 
pile up in the last node short of the destination. This in 
mourn could £111 up the buffers of all the nodes adjacent to 
the destination, creating a wall around the destination. If 
the destination node were a source of packets moving in the 
opposite direction, it would not clear its transmitting 
buffers. Soon the whole network would lock up. 

Another alternative to buffer storage allocation is to 
have the destination notify the source when it cannot accept 


a packet after the packet has arrived at the destination. 


f 
, 
fit 


This method requires the source to transmit these packets 
at least twice. Advanced allocation of reassembly buffers, 
resulting in occasional transmission delays, is more 
efficient than recovering from discarded packets. However, 
neither method is considered an adequate procedure for real- 
time or data-sharing users. 

Still another alternative to buffer storage allocation is 


the method of assigning special routes, based on status in- 
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formation received from adjacent nodes, and on traffic 
patterns encountered by the node over the last few seconds. 
Such a special route should not be used, however, in res- 
ponse to a rapid but short-lived increase in traffic flow. 
Therefore, measurements of the rate of change of traffic 
along each path are combined with a predefined interval 

of time, and alternative routing is established only if the 
traffic change persists. The algorithm has a number of 
rules: 

a. The routing selection is performed independently by 
each node, on the basis of information received from adja- 
Gent nodes and of traffic patterns; 

b. The algorithm guarantees that individual routing 
decisions possess global continuity for the network; 

c. Routing tables are always updated synchronously, 
with additional asynchronous updating when necessary; 

d. In selecting routes, the network is decomposed into 
' a union of identical and overlapping subnetworks with se- 
| parate routing computed for each subnetwork; 

e. For an unloaded net, a path is made which crosses 
the fewest nodes on the way to the destination; 

f. For a loaded net, traffic is diverted from fully 
Occupied links whenever possible; 

g.- Changes in routing occur only when a new traffic 
pattern is sustained for a minimum time; 


h. Additional paths can be established for a given 
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destination to allow individual packets to depart on sepa- 
rate links; 

i. Traffic flow on any link in a subnetwork can occur 
in Bey fone direction at a time (half-duplex operation) ; 

ipeebtrection of Elow in a link may change, but only 
mrtter the link remains unused for a short interval of time; 

k. The maximum allowed traffic through each node in a 
subnetwork is regulated so as to change slowly; this pro- 
Vides stability and allows adjustments for increased traf- 
mic flow: 

1. For each subnetwork, loops in routing are quickly 


detected and broken. 


=. THE CCITT STANDARD HOST INTERFACE-X.25 
CCITT recommendation X.25 is an international standard 
for the “interface between data terminal equipment and data 
circuit-termination for terminals operating in the packet 
node on public data networks". In other words, this inter- 
face is intended for use by host computers and other types 
of customer terminals which are intelligent enough to for- 
mat data in the way specified by the interface standard and 
to respond in the proper ways to the complex control signaling 
requirements of the interface. The interface is designed for 
multi-channel use; that is, it provides for communication 
with multiple independent user processes within a single 


maser terminal (computer). 
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The X.25 recommendation is structured into three inde- 
pendent parts as follows: 

Level 1 - the physical, electrical, functional, and 
procedural characteristics to establish, maintain and dis- 
connect the physical link between the data terminal equip- 
ment (computer) and the data communications equipment; 

Level 2 - the link control level for the interchange of 
data between the data terminal equipment and the network; 

Level 3 - the communication control level defines the 
formatting of information and control procedures used 
between the data terminal equipment and the network for the 
purpose of transferring user data through the network and 
for establishment of end-to-end connections. 

Figure 6.1 shows schematically the typical connection 
of data terminal equipment to a network and the physical 


point of interface where the standard is applicable. 
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Figure 6.2 displays the logical relation of the hierar- 


chical levels to the network components. 
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Figure 6.2 - Hierarchy of interface levels 


For level 1, the X.25 recommendation specifies the use 
of CCITT recommendation X.21, a general-purpose interface 
for synchronous operation in public data networks. For an 
interim period, the use of recommendation X.21l-bis (essen- 
tially the same as E1A RS-232) is approved. 

For level 2, a link access procedure is defined. It 
uses the principles and terminology of the High Level Data 


Link Control Procedure (HDLC) specified by the International 
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Standards Organization (ISO). This is a bit-oriented line 
discipline suitable for use in a variety of environments. 

Level 2 provides the function of error and flow control 
of the access link between the DTE and the network. Each 
frame has a check sequence to detect errors. Error frames 
are retransmitted when requested by the receiving end. Flow 
control is accomplished through sending of Receiver Ready 
(RR) commands or Receiver Not Ready (RNR) commands (buffer 
Storage allocation). 

Level 3 of K.25 defines the packet formats and control 
Procedures for exchange of information between a DTE and 
network. The type of service supported by the current X.25 
is the so-called virtual circuit service, which is a public 
service for virtual circuit switching. Establishment of a 
Virtual circuit is initiated from a calling DTE by a Call 
Request packet. The called DTE is notified that a "circuit" 
1s being established by an Incoming Call packet. Subse- 
meeentiy, the calling DTE is notified that the circuit is 
established by a Call Connected packet. Data packets are 
then exchanged between DTE's. Upon completion of the call, 
the circuit can be disestablished by either a DTE Clear 
Request packet or DCE Clear Indication packet, as appro- 
priate, followed by a Clear Confirmation packet response. 

The capability for multiplexing up to 4096 logical 
Channels (virtual circuits) on a single access link is also 


provided by X.25. Each logical channel can be used for 
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Virtual calls or a permanent virtual circuit. Each packet 
exchanged across the interface has it associated logical 
channel number identified. Each logical channel operates 
independently of others. The data packets are also each 
identified by a sequence number, which is used for flow 
control on individual logical channels. RNR packets are 
sed to stop transmission on a channel while RR packets 
permit transmission. The sequence numbering scheme may be 
based upon either modulo 8 for normal operation or modulo 
128 for extended transmission delay conditions. The se- 
quence numbers are then recycled every 8 or 128 packets, 
aS appropriate. Packet formats for the different types of 
packets are specified. Data packets are limited to a max- 
imum data field Neneene All networks must allow a maximum 
of 128 octets (8 bit byte), while some networks may also 
support maximum lengths of 16, 32, 64, 256, 516 and 1024 
octets or 255 octets on an exception basis. Thus, no 
packet assembly/disassembly capability is assumed by the 
' interface (DCE). Networks supporting the X.25 protocol 
are free to disassemble the octet data fields into still 


smaller packets for internal switching, if this is required. 


140 








PART 2 


DESIGN OF A NETWORK FOR THE NPS 


TIME SHARING SYSTEM 
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Vii. | PROBLEM DEE EN EGION 


ee LHe PRESENT NPS CENTRAL COMPUTER FACILITY 

The W. R. Church Computer Center, located in the first 
floor of Ingersoll Hall, is the organizational unit of the 
NPS which has the responsbility of providing campus-wide 
computer services. Its services are available to all faculty, 
staff, and students of the school for use in connection with 
instruction, research, or administrative activities. The 
center offers two modes of operational service, i1.é., batch- 
processing and general purpose multi-access time-sharing. 

The present equipment, based on an IBM/360, includes 
two model 67 processing units; four different levels of 
storage, including 2 M bytes of core, 4 M bytes on a drun, 
24 disk drives with 29 M bytes each and 8 disk drives with 
100 M bytes each and 9 magnetic tape units; two high-speed 
plotters, fifty remote hard-copy and video terminals, and 
an IBM 2550 Graphical Display Unit. The two processors 
are identical and, by means of a configuration control unit, 
can access directly, or control, all components of the 
system including core storage modules, input/output con- 
trollers and devices. 

The allocation of resources of the system to each CPU, 
msing the configuration control unit, is static and mutual- 
exclusive, that is, one unit can only be used by just one 
CPU at a time; thus, the physical resources of the system 


are divided between the two CPU's. 
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Typically, during working hours, the computer center 
operates with two independent systems - one CPU under the 
OS/360-MVT operating system, runs the batch processing and 
the other one, under CP-67 with CMS (Cambridge Monitor 
System) runs multi-access time-sharing. The batch envir- 
onment is a punched card oriented one in which jobs are 
typically submitted to the system by means of a deck of 
cards. There is little, if any, coupling between the two 
systems (batch and time-sharing); there is no way of auto- 
matically transfering one application program from one en- 
vironment to the other; for example, if one edits and tests 
a program in the time-sharing system and wants to run it in 
the batch system, one has to first have the program punched 
and then has to submit the card deck to the batch processor. 

With the exception of remote terminals and modems, all 
the hardware is concentrated in Ingersoll Hall. Thus, the 
input card decks have to be hand carried across the campus 
in order to submit a job in the batch mode; also, printed 
outputs have to be picked up in the computer center, even 
printer outputs of the time-sharing system. 

The center possesses a wide variety of software resources. 
In the batch environment, under OS/360-MVT, the following 
language processors are currently available: 

a. FORTRAN IV G; 

De peuCRTRAN LV iH: 


c. WATFIV; 
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ae Piy/ li; 

e. PL/C; 

f. Assembler 360; 
g. ANS COBOL; 

h. GPSSV; 

1. SIMSCRIPT; 

j. ALGOL W; 


k. WATBOL; 


1. PLM; 
m. FORMAC; 
n. BASIC. 


In addition to the software facilities which are part of 
OS/360-MVT and those routines embedded in each language 
processor, the center maintains a large library of programs 
For public use. The library is composed of programs from 
many origins: IBM "Scientific Subroutine Packages", computer 
center-supplied programs, faculty and student-supplied rou- 
tines, International Mathematical and Statistical Library and 
others. Some large applications packages which, in effect, 
provide special language capabilities are available in the 
eibrary: 

a. Mathematical Program System (MPS); 

b. Biomedical Statistics Packages; 

c. Electronic Circuit Analysis Program (ECAP); 

dad. Digital Simulation Language (DSL); 


e. Linear Systems Analysis (LISA); 
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f. Matrix Language (MATLAL) ; 
g. SORT/MERGE (IBM) ; 
h. Continuous System Modeling Program (CSMPIITI) ; 
i. Statistical Package for the Social Sciences (SPSS); 
j.-  SNAP/IEDA, Introduction to Explanatory Data Analysis; 
k. Eigensystem Package (EISPACK) ; 
1. Linear Systems Package (LINSYS) ; 
m. Function Package (FUNPACK). 
The public library stores programs in three forms: 
a. Source programs-those in high level languages; 
b. Object programs-those compiled but not directly 
executable programs; 
| c. Load programs-those which can be directly executable. 
In addition to the public library, the center supports 
private libraries, which are programs with access limited 
to one or few users and which are kept on-line for future 
use by their owners. 
The time-sharing environment allows the use of the 


following languages: 


- —, -~+-— 


a. FORTRAN G; 

b. Assembler 360; 
Gs MEAS IC; 

d. ALGOL W; 

e. PLM; 


£. APL. 
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The public library of CP/CMS is divided in three parts: 

a. System library (SYSLIB), which is the set of software 
facilities embedded in CP/CMS; 

b. SSPLIB, which comprises FORTRAN scientific routines 
in object language form; 

c. IMSL which is also in object language form. 

The APL interpreter does have an extensive library; also 
the APL processor is able to put in the user's terminal 
explanatory notes about the use of its library programs. 

Users of the time-sharing systems are classified into 
two categories: general users and private users. General 
users share common disk File storage; the files of general 
users are stored in the space allocated to the terminal 
where they were entered; a general user can only access 
files in the terminal where they were created. 

Private users have private disk space of fixed size, which 
is usually 256 K bytes. The space of a private user cannot 
be accessed by other users, unless they know his identifica- 
tion and password. 

The time-sharing system has some Tektronix 4012 Display 
Terminals which have the capability for interactive graphical 
work. There 1S no capability of off-line plotting, since 


the plotters are used with the batch system. 


®. THE NEED FOR A NEW SYSTEM 
One of the first steps in planning the substitution of a 


system should be to try to investigate the reasons for the 
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change. This has two objects; the first is to make sure 
that the change is really necessary; the second is to learn 
from past experience and, consequently, direct the new de- 
Sign in the right way, incorporating the missing and needed 
features and SU oucing the flaws of the old system. 

The Future Computer Planning Committee at the NPS has 
conducted two surveys, regarding user requirements. Based 
on the analysis of those surveys and another one performed 
by a student, the justification for a new computer system 
is summarized below. From these einatnes some of the goals 
and requirements for the new system can be readily derived. 

ie Rneliabrlicy 
The present system has been operating for more than 
ten years. Its failure rate in recent years has been un- 
acceptable. 

2. Needfor a Powerful Batch Computer 

The school has responsibilities for conducting ad- 
vanced education and research and requires powerful com- 
puters to support these goals. Science and technology have 
grown to a level of sophistication which demands enormous 
computational power. Even today's super computers are still 
inadequate to deal with current and future computational 
demands. For example, going from a two dimensional aerody- 
namic simulation-the current state-of-the-art capability- 
to a more realistic three dimensional formulation increases 


the computational demands by a factor of one hundred. 
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Doubling the resolution of today's climate models, already 
taxing the current super computers, increases the compu- 
tational demands by a factor of ten. Many other examples 
can be found in research areas of interest for the school 
such as control engineering, operation research, signal 
processing, physics, oceanography, etc. The school's batch 
system is very modest when compared to the capabilities of 
Super computers. Thus, the school is already behind in com- 
putational capability required for today's research needs. 

The analyses that were conducted have shown that 
the batch system performance was adequate for most of the 
educational needs, but it lacks capability to handle many 
of the school's research projects. Students submit the 
greatest number of jobs. The percentage of CPU hours used 
by students is much lower than their percentage of jobs. 

In contrast, the faculty has a much higher percentage of 
CPU time than its fraction of jobs. It was found that 
faculty research projects are large and tend to be CPU 
bound /Ref. 32/7. 

The study of aggregate data concluded that more than 
half of the jobs are executed in less than 10 seconds and 
that 80% of the jobs use less than 150 k bytes of memory 
/Ref. 32/7. Clearly, these numbers are heavily influenced by 
Student jobs. In contrast, an entirely different situation 
is indicated with regard to research computational require- 


ments. 
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The following conclusions were derived in rey Seach 
of the data provided by the principal batch processing 
users (researchers) /Ref. 39/: 

a. the school needs a computer at least 10 times 
faster in processor and memory speed than the IBM 360/67; 

b. a computer with at least 4 times the present 
batch system memory capacity will be required; 

c. other capabilities which are required are gra- 
phics terminals and a good data base system; 

d. research eee ess 1s inhibited, grant opportuni- 
ties are threatened and NPS research is becoming less com- 
petitive due to inadequate computing services. 

3. iInadequacy of the Present Time-Sharing System 

The survey that was conducted among major time- 
sharing users did not enable the derivation of a functional 
specification, as the survey of batch users did, because 
of the inconsistency of the data collected /Ref. 40/. 
Nevertheless, important findings and conclusions were de- 
rived. Also, the previously mentioned analysis conducted 
by a student presents some facts and comments extracted from 
interviews of representative users and academic officials 
/Ref. 327. 

The aggregate of those two studies regarding the 
time-sharing system portrays the present situation: 

a. Since 1975 the usage of the time-sharing system, 


Measured in average CPU hours used, has increased steadily; 
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b. Presently, response time under CP/CMS is consi- 
dered inadequate when more than a few terminals are in use 
mours of intense use) ; 

c. There is now a great demand for terminals, even 
early in the quarter; students are standing in line to use 
terminals; the demand on weekends is almost as high; 

d. Sometimes it is not possible to use a terminal 
because of the inability of making a connection via phone 
company lines (the way almost all remote terminals are 
Brnected to the system); 

e. The use of APL is intense; 

f. Professors are generating a greatly increasing 
time sharing load due to course work; 

g. There is a need for communication between the 
batch and the time-sharing system; 

h. There is a need for faster terminals; 

1. Electronics Engineering, Mathematics, Operations 
Research and Aeronautics all request availability of remote 
plotters so that graphics routines can be used in an inter- 
active mode. 

On the whole, these facts show the inadequacy of the 
present time-sharing system. In particular, a), f£) and i) 
Suggest that the usage of time-sharing will increase if the 
facilities are provided. But, b), c) and da) show that the 
present system is already saturated. From b) and d) it can 


be concluded that the addition of more terminals would degrade 
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even more the performance of the system and, thus, is not 


recommended. 


In contrast with the batch system, in time sharing 


the major users do not represent a significant portion of 


the system load. The major time sharing users account for 
/Ref. 40/: 
a. 7% of the total number of uSers; 


b. 113 of the total number of Sessions; 

c. 16% of the total terminal time; 

d. the average time per session of the major users 
is 1.4 times that of all users; 

e. only 37% of the major users polled have more 
than 256 k bytes of private disk space. 

These figures are of great importance in view of the 
planning for the new time sharing system. They suggest that 
the majority of time sharing usage is dedicated to educa- 


tional purposes versus research. 


=. SCOPE OF THE DESIGN 

The design proposed by the Future Computer Planning 
Committee is toward a system able to work in two modes of 
operation: batch and time~sharing. Each mode presents 
different requirements. The functional specifications for 
the batch computer have already been derived. 

In this analysis the existence of a batch system is 
assumed. The design of this batch system is beyond the scope 


Br this study. 
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The objectives of this work are: 

a. Development of a distributed system approach for 
the future time sharing system of the NPS; 

b. Presentation and analysis of some distributed 
architectures which can satisfy time sharing requirements; 

c. Integration of batch and time sharing into a 
unified system. 

d. Presentation of a data communication structure 


for support of time sharing and remote batch processing. 
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VIII. DESIGN BACKGROUND 


A. MAN-MACHINE INTERFACE 

One of the major concerns of the design or selection of an 
interactive computer system should be the interface to the 
users, that is, the appearance of the system to the user. 
This aspect of the operating system of a time sharing facility 
will determine its usefulness. Good principles of design 
in this area, also called "human engineering", should be also 
present in the software facilities of general use such as 
library programs. 

There are two properties of an interactive system which 
are essential to the usefulness of the system but which com- 
pete with each other to some extent. These properties are: 
a) effectiveness and b) simplicity of use. By effective- 
ness 1t is meant the capability to solve the required pro- 
blems in a reasonable time. To provide this power there 
must be generality and flexibility. The simplicity of an 
interactive system is inversely dependent on the quantity 

/ and complexity of actions required by the user /Ref . 43/. 

f It is easy to see how a requirement of simplicity com- 
petes with a desire for effectiveness, since as more gener- 
ality and flexibility are put in a system, more choices 
(actions) must be made and more complex information must be 
given by the user to the system to tailor its power to the 
particular problem at hand. The problem of resolving the 


conflict between generality and flexibility versus simplicity 
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of use is one of the significant factors in the selection 
or design of an interactive system. 

One way of resolving conflicting objectives of simpli- 
city of use, generality and flexibility at the expense of 
increased software, is through the concept of levels of 
mnvisibility /Ref. Wa Operating systems designed under 
this concept present to the user a linear structure of soft- 
ware modules. This structure is often represented as a 
ring structure consisting of successive levels or layers 
as in Figure 8.1. Each successive layer conceals details 
meaningful only to lower levels. Thus, modules of inner 
levels are more general and flexible but less simple to 
use. At the highest level the system has most of the commands 
needed by the common user in pre-defined modules which are 
very simple to use. A skilled user, on the other hand, may 
shape his own high-level utilities, using the lower level 
modules. 

There are several general design principles that should 
be embodied in a user oriented interactive system. Some of 
them are /Ref. 43/: 


1. self-explanatory; 


2. self-help user assistance}; 
3. simple interface with user; 
4. interaction by anticipation; 


5. optional verbosity. 
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INCREASED SIMPLICITY 


DECREASED GENERALITY 
AND FLEXIBILITY 


Figure 8.1 - Levels of abstraction 


A self-explanatory system is one which displays to the 
user sufficient explanatory information about the process 
being carried out to enable him to carry on without refer- 
ence to some external source of explanation (the system's 
manual, for example). This can be accomplished by dis- 
playing certain basic items and allowing the user to ask 
for optional additional tutorial material. 

A self-help system provides checking of user inputs to 
the system and provides reminders or instructional displays 
On user request. Input items are checked for validity or 
reasonableness, and if appropriate, a diagnostic message is 
sent to the user and the system awaits his further instruc- 
eions. 

Simple interfacing with the user is accomplished by en- 
suring that the actions required of him are short, simple 
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and obvious. One aspect of the interface is the grouping 

of options presented. For example, if many options are 
available, it is probably easier for an on-line user to 

make his choice if the options can be presented in subgroups 
of five to seven items. 

A desirable feature of any on-line system is interaction 
by anticipation. By this it is meant that all possible 
desires of a user are anticipated (hopefully) and choices are 
presented which include all those possibilities. This method 
allows the user to select a desired option rather than specity 
that option by entering a correctly spelled command. With 
a teletype, typewriter or CRT terminal the options could be 

' numbered seguentially as they are printed, and the user would 
select by simply entering the number corresponding to his 
choice. The problem of diagnostic messages is alleviated 
by employing interaction by anticipation. 

An interactive system featuring optional verbosity can 
be considered as a system with two levels of detail in the 
| anterface with the user. For the novice or first-time user, 
the interface will contain detailed explanations to insure 
ieee the user understands what he is expected to do and what 
the computer is doing. On the other hand, an experienced 
user might choose the mode of operation with few or no ex- 
planations and abbreviated communications (concise messages) 
both to and from the computer. This optional verbosity 


is especially desirable with teletypes or typewriter 
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terminals which have slow printing rates, making lengthy 
messages time consuming and boring the experienced user. 
With a CRT terminal with fast display the verbosity option 
is not essential in messages coming from the computer but 
it is still desirable in the user to computer direction. 
Another important factor to be considered is the dis- 
tinction between reversible and nonreversible requests. 
A reversible request is one which can easily (without signi- 
ficant computer utilization) be reversed. A nonreversible 
request cannot be reversed easily (uses a significant 
amount of computer time) or even cannot be reversed at all 
(for example, inputs are lost or modified in the process). 
Reversible requests should be carried out immediately by 
the computer. Nonreversible requests, on the other side, 
should require confirmation by the user; the system should 
send a message calling the attention of the user by echoing 
the command or explaining the actions to be performed upon 
execution of the command; after confirmation by the user 
the required action will take place. This procedure can 
avoid common disasters such as erasure of the wrong file, 


unwanted modification of a data base, etc. 


B. CHARACTERIZATION OF THE TIME SHARING AT NPS 
1. Characteristics of the Work 
The main purpose of the time sharing system at NPS 


is to provide computational power accessible to a multitude 
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of users, students, faculty and staff, in an interactive 
way. Thus, users go to the terminal looking for a pro- 
blem solver, a number cruncher or a clerical helper. Each 
user submits a specific and different job. Transactions 
cannot be defined or characterized as opposed to a business 
information system where they are standardized. Viewed 
this way the system can be characterized as a computational 
system. 

There is little interaction among users and conse- 
quently little terminal to terminal communication. But this 
capability should not be ruled out in the new system because 
group projects may require communication among the group 
components for coordination; this communication may be per- 
formed either directly on a terminal to terminal basis or 
by means of a "mail box" where users drop messages which 
are stored until they can be delivered when their addressed 
users log on the system. 

The majority of communication is in both directions 
_ between user and processor. Here delays must be minimized 

Since they incur degeneration of the system response time. 
Typical usage of the time sharing includes: 
a. program edit; 
b. text edit (word processing) ; 
c. debugging of programs; 
c. execution of application programs from the system 


library or implemented by the user. 
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2. Characteristics of the Software Resources 
The software resources of the time sharing system 
are the operating system and the data bases. The operating 
system manages the use of the system resources and inter- 
faces the users to the system. The operating system incor- 
porates a set of software utilities callable through a 
command language. 


Typical software utilities of the operating system 


include: 
a. commands for execution of specific tasks; 
b. commands for coordination of tasks; 
c. language processors; 
d. line editor; 


e. text editor; 

f. debugger. 

The data bases incorporate application software and 
data. They can be divided into: 

a.. System library-the set of software utilities 
not included in the Re system and intended for public 
use on a read-only basis; the utilities consist typically of 
programs and routines in high level or machine language which 
can be incorporated into the user's application software, ex- 
tensive software packages similar to language processors for 
specific applications and software helps and files of data 


for specific use; 
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b. User library-collection of application programs 
and files of data kept on line by the users for future use; 
typically they have restricted access and are not intended 
for public use. 

3. Analysis of Data 

The computer center compiles statistics of the time 
sharing system every month. A set of these data covering 
the period from February of 1977 to January of 1978 was 
studied. The data was furnished in aggregated form; a pre- 
cise definition of the terms used, the calculation process 
of the Beers and the way the measures are performed 
were not available. So, reasonable assumptions were made 
regarding these missing definitions. Time lost due to 
system failures is not included in the statistics; the 
assumption is that its influence is small. 

The analysis which follows is useful for qualitative 
characterization of the problem; for the reasons given 
above an extrapolation of parameters as a basis for de- 
Signing the new system would be dangerous. A complete anal- 
ysis, using the raw data gathered by the CP/CMS usage moni- 
mor, is recommended for this purpose. 

a. Volume of Use 

Table 8.1 displays three measures of the volume 
of use of the system: 
1. Number of sessions-total number of sessions 


performed during one month at all terminals. 
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2. Number of users-total numer of different 
users which have used the system during one month; 

3. Terminal time-total time that all terminals 
in the system were active in one month. 

The data are displayed in absolute value and 
in the form of percentage increments relative to the month 


Be February 1977. 


Number of Sessions Number of users Terminal time 
Month ABSLT INCRMNT ABSLT INCRMNT ABSLT INCRMNT 
(3) (3) (HOURS) (3) 
PEB/77 S77 0 225 0 2174 ) 
MAR/77 556 6 -6.2 230 Avie 234.5 O32 
APR/77 4328 Zoe 256 Woes 2833 3053 
MAY /77 4461 B2ak 239 Ge 2828 Sop ade 
SUL/77 ES IES -14.7 Leg -16.0 20.07 -8.0 
AUG/77 3514 4.1 229 ales: Zn lie) 
OCT //7 3299 -2.3 248 iO eeZ so -14.8 
NOV/77 By 2/ 69.6 309 5/23 3858 (eae 
DEC/77 3314 =| .8 255 lies 2346 7.9 
JAN/78 3828 13.4 286 TAG ea 262 -0.6 


Table 8.1 - Volume of time sharing use 


Data referring to June 1977 were missing and 
that of September 1977 were inconsistent and discarded. 
An examination of the table shows: 
1. By every measure, the volume of use increased 


during the period examined, while not steadily, but in ascend- 
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ing cycles; in fact, the application of linear regression 
analysis to the three measure discloses: 
-~ an average growth in the number of sessions 
of 55.9 sessions per month 
-~ an average growth in the number of active 
users of 6.4 users per month 
-~ an average growth in the terminal time of 
15.2 hours per month; 

2. The cycles match approximately the quarters 
of the academic year with the greatest use occuring in the 
second month of every quarter. 

b. Terminal Usage 

Table 8.2 displays several data relative to 
terminal usage: 

l. Sessions per hour-average of the number of 
sessions derived by dividing the total number of sessions 
by the total number of operational hours of the system in 
one month; it is presented in absolute value and incre- 
mental form relative to February of 1977; 

2. Session time-average duration of one ter- 
minal session in the overall system; 

3. Interarrival time-average time between two 
Successive arrivals to one terminal; 

4. Terminal utilization factor-average fraction 
of terminals in use or average terminal capacity that was 


utilized; it was calculated in two ways: 
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MONTH 


FEB/77 
MAR/77 
APR/77 
MAY /77 
JUL/77 
AUG/77 
ECT /77 
NOV/77 
DEC/77 


JAN/78 


creased 


of 0.09 





~ method 1 - the session time was divided by 
the interarrival time; 

- method 2 - the total time that all terminals 

in the system were active was divided by the 


total number of operational hours of the 


system times the number of terminals. 


SE oomehs se Bk 


HOUR TERMINAL 
SESSION INTERARRIVAL UTILIZATION 
ABSLT INCRMNT TIME TIME FACTOR 
(3) (MIN) (MIN) METHOD METHOD 
1 2 
520 0 38.26 122 0.32 0.19 
IEG ~26.7 43.8 167 0.26 0.16 
E625 EO. 0 39.3 eZ O35 0.22 
16.3 Sei 38.0 114 Os O21 
eee ~18.7 eg L53 OeZ7] O27 
iaIeaG ~22.7 Sire c 159 OQ. 24 O22 
i ly ~25.3 Sn 7 166 0220 Oz 13 
L6.9 27 40.4 Oy 0.38 O23 
12.9 ~14,.0 a2 5 146 0.29 0.18 
ins 5 ~10.0 S329 a7 Oe25 Oils 
Table 8.2 ~ Terminal use 


Analysis of Table 8.2 discloses: 
1. The average number of sessions per hour de- 
slightly during the period; the trend was a decrease 


sessions per hour per month by a linear regression 


computation; considering that the number of sessions in- 
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creased in the same period, this shows that the adoption of 
extended hours of operation by the computer center overcompen- 
sated for the growth in usage; 

2. The monthly average session duration is an 
almost constant characteristic of the time sharing usage at 
NPS; indeed, the figures in the column "session time" have an 
average of 39 and a standard deviation of 3.4, which is 
small compared to the mean; it is worth noting that these 
numbers do not give information about the distribution of 
session times; beyond this monthly mean nothing is known, 
because the data were aggregated; an average of forty min- 
utes seems brief oe the duration of a typical student 
session; other users may account for enough short sessions 
to lower the mean; caution should be exercised in uSing 
these figures. 

3. If the column of absolute values of number 
of sessions per hous is ordered according to the eee 
figures, the corresponding interarrival times will be per- 
fectly ordered in decreasing values; this shows a good de- 
gree of consistency between these columns of data; never- 
theless, for a system which has user terminal queues, the 
interarrival times displayed are large compared to the 
session times; possible explanations for this fact are given 
below; 

Peeve laZat HON mace Ons sOmuternanaloucalellated 


by mode method 2 are consistently lower than those calculated 
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mode method 2, thus making the assumption of small influence 
of system failures suspect; an important fact is that both 
methods yield figures which are low in relation to queue 
sizes observed at terminals; the above may be explained by 
one or more of the following: 

- there are many underutilized terminals (this 
is readily noted in the computer center data), 
which may be due to: 

- low availability (combination of failure 
rate and repair time) 

- many private terminals which have low usage 

— vimeeen’ public termana ls - 

- there is little '‘diffusion' of users, i.e., 
they usually do not look for:vacant terminals 
in other than the site near their work place; 

| 1£ terminals are not evenly distributed around 
the campus some terminals may be subject toa 
high demand because they serve a more active 
community; 

- the distribution of arrival times and service 
times (average session time) have particular- 
ities which lead to congestion; for example, 
the demand for terminals may be concentrated 
in a small fraction of the system operational 
time or the interarrival times happen to be 


short in a period of large service times. 
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CemecereU Uti lazation 
Table 8.3 condenses the following figures rela- 
tive to CPU utilization in the CP/CMS system at the school: 
1. CPU time-the sum of virtual CPU time uti- 
lized by all users during one month; it is presented in 
absolute value and incremental form relative to February 
eee 9 7 7 > 
2. CPU time per session-the average virtual 
CPU time spent in one terminal session; 
3. CPU time per user-the average virtual CPU 
time utilized by each user during one month; 
4. CPU utilization factor-average fraction of 
the operational time that the CPU served the users or 
average CPU capacity that was utilized; it is presented in 
two ways: 
- basic-the total virtual. CPU time (simply CPU 
time above) was divided by the total system 
G@perational time, both during one month 

- adjusted-an arbitrary overhead factor equal 
to 1.25 (realistic) was applied to the CPU 
time and another arbitrary reliability factor 
of 0.9 was applied to the operational time to 
account for failures during operational hours; 
then the calculation above was performed; 

5. Terminal time per CPU time-the total time 


that terminals were active was divided by the CPU time; this 
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16 the number of seconds of terminal session which corres- 

ponded to one second of CPU time in the particular month; 
6. Active terminals-expected value of active 

terminals obtained by multiplying the total number of ter- 


minals by the terminal utilization factor (method 1). 


CPU CPU TERMINAL 
TIME TIME CPU TIME 
CPU TIME PER PER UTILIZATION PER 
MONTH ABSLT INCRMNT SESSION USER FACTOR CPy ACTIVE 
(MIN) (3) (SEC) (SEC)BASIC ADJUSTED TIME TERMINALS 
FEB/77 4058 0 72 1082 0.30 £0.42 32.1 16.0 
Ary 77 6759 66.6 Ipeeeli7es Onc hOese 20.6 13.0 
 APR/77 5697 40.4 ji) MEI lsne ley) Were ye 29.8 es 
MAY/77 5451 34.3 fae ieee) oles oat ile lees 
Bry 77) 3333 -17.9 69 1086 0.24 40.33 36.0 aa 
AUG/77 3389 -16.5 57 888 0.19 0.26 Boal 12.0 
OCT/77 3304 -18.6 60 799 0.19 0.26 ae). G 10.0 
NOV/77 5211 28.4 oie o826. sOmeG 44.4 19.0 
P DEC/77 5564 37.1 100 1309 0.36 0.50 Dine aes 
Man /78 2381 -41.3 37 497 0.14 0.19 Bas 12 


Table 823 — CPU utilization 


Analysis of Table 8.3 follows: 
1. The total CPU time demanded per month varied 
considerably during the period and showed a tendency to de- 


crease; by a linear regression calculation this decrease is 
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in average 195.6 minutes per month; this is in contrast with 
the volume of time sharing use which increased; the ex- 
planation may be that the increase in volume was due to 

the entry of novice users demanding little computing power; 

2. The CPU time per session and the CPU time 
per user during each month are modest, thus showing that 
the time sharing system is used for simple problems; 

3. The CPU utilization factor is low even when 
corrected by the factors estimated above (adjusted value) ; 
to justify the claim that the response time is inadequate 
when more than a few terminals the following factors should 
be considered: 

- most probably the system is I/O bound-the 
bottleneck is not the CPU, but some 1/0 or 
storage device 

- there is a large amount of overhead in the 
Operating system which is larger than the 
25% assumed above 

- the use of the system is concentrated ina 
small portion of the operational time 

- the MTTR (mean time to repair) is larger than 
the 10% of the operational time assumed above 

- the data are not reliable; 

4. A comparison of the last two columns shows 
that, in terms of CPU, the system has on the average a rea- 


sonable margin of capacity. 
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d. Frequency of Use of the System 
The average number of sessions per user is dis- 
played in Table 8.4. The ‘average user' has around 16 
sessions per month. The trend in the period was a de- 
crease of 0.17 sessions per user per month, a result which 
again may be explained by the entry of new users with 


light usage of the system, probably students. 


MONTH SESo LONG] PER 
USER 
PEB/ 77 130 
MAR/77 136 
APR/77 Toe 
MAY /77 Sle ae: 
JUL/77 Tor y2 
AUG/77 5 
OCT/77 alk oe: 
NOV/77 Ue 
BEG] / 7 i350 
JAN/78 13.4 
Table 8.4 - Average number of sessions per user 


Figure 8.2 provides a better idea of the fre- 
quency of use of the system; the community of active users 
each month was grouped according to the number of terminal 
Sessions held by each. The communities are not necessarily 


the same every month; also, users may change classes from 


INS 8. 





one month to another. It is possible to divide arbitrarily 
the users of each month into three classes: 

- constant users, which have more than 16 
sessions and represent approximately 30% of 
the total 

- regular users, which have more than 4 and 16 
or less sessions and amount to approximately 
S5400L the total 

- sporadic users, which have 4 or less sessions 
and are around 35% of the EOtal., 

The relative position of the curves in Figure 8.2 
suggest that the monthly distribution of users and sessions 
is approximately the same for all months. This fact is 
corroborated by Figure 8.3, which shows the cumulative dis- 

' tribution of users versus sessions for the months of October, 
November and December of 1977. The existence of constant 
users with a large fraction of the total number of sessions 
and of sporadic users with a light fraction but representing 
a Significant portion of the number of users is obvious: 

- 10% of users held more than 30% of the 
sessions 

~ 30% of users held less than 5% of the session. 

e. Distribution of the Workload by the Time of the 

Day 

Figure 8.4 displays the curve of the average of 


mae maximum number of active users in slices of 30 minutes 
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during a day calculated over the entire period studied with 
the exception of June and September of LOT Ties curve 
was obtained by summing the maximum number of simultaneous 
active users in each half-hour slice over the period and 
dividing this sum by the total number of periods for each 
elice. The calculation did not include failures, but, if 

a uniform distribution of failures along the day is assumed, 
this factor would not change the shape of the curve. 

Three local maximums are recognized: 

- around 1030 in the morning 

- around 1500 in the afternoon, which is.also 

the absolute maximum 

- around 2030 in the evening. 

Two local minimums are present: 

- around 1200 

- around 1830. 

The afternoon peak is equal to 15.1 and lasts 
longer than the morning peak. 

The dashed line divides the curve into three 
regions. The upper part represents 50% of the system working 
hours and is the period where most of the work is performed. 
In this period, which extends from 0930 to 1700, the expected 
value of the maximum number of simultaneous active users is 
always larger than ll. 

The steep slope of the curve by the end of the 


Operational time suggests that users could extend their work 
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beyond 2200 if the system were available. 

The curve of Figure 8.4 suggests that the ter- 
Minal utilization factors calculated by method 2 in Table 
8.2 are reasonable because they give expected values between 
6 and 13 for the number of connected terminals, which is 
consistent with the curve; if the values provided by method 
fd are used, this does not occur. 

The maximum number of simultaneous users logged 
On the system in the last quarter of 1977 was 26. The max- 
imum recorded in the examined period was 35 simultaneous 


moers. 


C. DESIGN GOALS 

Based on the characterization of the problem and the 
Characteristics of the NPS environment, a set of desirable 
attributes for the new system is developed in the following. 

ie Aecessibility 

The computer power should be spread throughout the 

ampus where the need for it exists. Terminals for the time 
Sharing system should be evenly distributed; points for 
ntry of jobs and reception of outputs of the batch system 
Should be dispersed around the campus to avoid crossing the 
campus to do computer work. 
2. Modularity 

The average life span of a computer system of this 
seype is about 8 years. In this time the needs of the NPS 


nm terms of computing power may increase considerably in 
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volume. The system should have an architecture which can 
easily absorb this growth by allowing expansion to occur 
smoothly. 


3. Adaptiveness or Evolvability 


The school is a research center. The nature of the 


research in the government and in the military is very dyna- 


Mic, reflecting changes in national policies, and world con- 
ditions. Also, technology is progressing at unprecedented 
rates, making predictions very difficult. The new system 
should be adaptive, i.e., it should be adaptable to changes 
i needs and technology. 
4. Reliability 

A large part of research and academic work depends 
on the use of the computer for its Son leeone. The system 
measure of interest to the uSer is availability, which is 
the expected fraction of the operational time in which the 
system is available to its users. Availability is a func- 
tion of the MTBF (mean time between failures) which is a 
Measure of reliability, and the MTTR (mean time to repair) 


which depends on many administrative and technical factors. 


‘The MTTR plays an important role in the system availability 


but, at the time of initial system design, there is little 


Or no control of it. On the other hand, reliability is 
strongly influenced by the system architecture. Thus, for 
the purpose of comparing architectures, reliability is more 


Meaningful. Criteria for reliability must be established. 
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The following criteria meets the purpose of this study: the 
probability of no failure in a specified time interval such 
that would put one or more sites out of operation. The goal 
for the NPS time sharing is a highly reliable system. 

>. Performance or Effectiveness 

Performance or effectiveness is the degree to which 
the system meets the demands of its assigned tasks. In time 
sharing systems the performance is mainly measured by the 
system's response time, i.e., the time elapsed between the 
moment that a user submits a request for service and the 
reception of output. Loosely speaking, the system should 
present an adequate response time to users' requests; res- 
ponse time is subjectively Nueces be adequate when it 
meets user expectation, which depends on the complexity of 
the request made. 

To handle the multitude of different needs at tne 
NPS with adequate quality of service, the system should 
exhibit generality and flexibility. Generality relates to 
the ability of handling diverse problems and flexibility to 
adapt to particular problems presented by the users. 

Another important factor in system performance is 
the ability of graceful degradation, which is a scheme 
whereby performance will be reduced as elements of the sys- 
tem fail, avoiding a total collapse, up to a limit which 
depends on the type and combination of the failed elements. 


The NPS system should provide graceful degradation to allow 
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work to continue during failure periods. 
6. Simplicity of Use 
The existence of a significant number of users with 
a reduced number of sessions each month was substantiated. 
Tt is reasonable to assume that many users are permanently 
included in this class of light usage (this fact cannot be 
deduced from the data). Such users typically do not develop 
great intimacy with the system. Also, many students are in- 
troduced to the system every quarter. Thus, the system 
should provide simplicity of use for fast learning. 
7. Ease of Operation 
A corollary of the simplicity of use goals is that 
users should be required to have only the minimal physical 
contact with equipment and should not be involved in opera- 
tions. Thus, professional operators should assist in the 
use of the system. This also enhances the availability of 
the system. But, practical and cost considerations dictate 
the minimization of the number of operators. Thus, the new 
system should exhibit the characteristic of ease of operation 


in the sense that it should not require many operators. 


D. DESIGN CONSTRAINT 
The only design constraint to be imposed is proven tech- 


nology. This means that the design should be based on com- 





mercially available products, hardware or software. Constru- 


ction of special hardware, large software development and 
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revolutionary technologies will not be considered. Never- 
theless, small modifications in hardware equipments and 
in software products may be taken into account. 

Cost constraints were not explicitly identified. Never- 


theless, cost considerations will be considered. 


E. REQUIREMENTS 
Many requirements for the time sharing system were 
identified in this and the last chapters. These and others 
will be briefly listed in the following: 
1. High Level Languages 
2 ait 
Db. ebAS LC 
Cc. FORTRAN 
d. PASCAL 
e. A system programming language. 
2. Debugging Facility 
Most of the debugging facilities are embedded in the 
language processors. In addition to these the system should 
provide an interactive debugger with general application to 
facilitate program development. 
3. Line Editor 
A line editor of general application should be pro- 
vided for the editing of programs in any of the languages 
offered. 
4. Text Editor 


Editing of text is needed for preparation of reports 
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and theses. Provisions for handling mathematical format 
should be considered (even if all mathematical symbols are 


not included in the character set). 


Seencegrated Batch-Time Sharing 


A program interactively developed in the time sharing 
mode should run in batch mode. Conversely, a program entered 
in batch mode should be callable from time sharing. 

6. System Library 
A set of high level and pre-compiled routines to be 
read and incorporated into users' programs. 
7. User Library 
Means of storing users’ files for future use. 
8. Calculator Mode 

The calculator mode of terminal operation pertains 
to an algebraic language of easy use with extended precision, 
which is intended for solving computational problems in the 
Manner of a hand-held or desk calculator. 

9. Mail System 

A system for exchange of messages should be consi- 
‘dered for communication among users even if the recipient is 
not logged on the system at the time that the message is 

Bent. 
10. Data Base Management 
A data age Management system for interactive use 
“should be provided. The exact needs in this area have yet 


to be studied; the NSA, AS and administrative departments 
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need this capability. 
ll. Graphics Terminals 
There is the need for graphics terminals for inter- 
active graphical work. This usage is typically analysis of 
curves for engineering, mathematical and statistical work. 
Terminals based on a DVST (direct view storage tube) seem 
to be adequate for most of these purposes; they have the 
advantage that they do not require refreshing, which alle- 
viates computer processing considerably. They should also 
work in ‘character mode’ for common use and have a means 
for copying screen contents. 
12. Command Language 
This 1S an integral part 2 ee time sharing system 
and pertains to the set of commands and rules available to 
the user for utilization of the system resources. It should 
exhibit good characteristics of human engineering as dictated 
by the goal of simplicity of use, while allowing complex 
operations as dictated by the performance goal. 
ieee .SeCurLtey 
Security in the computer field is viewed in many as- 


pects. Those aspects relative to classified work will not 


'-be addressed; they should be studied in light of the specific 


regulations to determine the suitability of the system and 
the methods needed for implementing this kind of work. 
Security may be viewed as involving authority of 


access, that is, as a question of establishing who (users, 
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processes) can be allowed to access what (files, processes, 
systems resources in general). Security may be externally 
imposed, as in the protection of files from unwarrented 
examination (privacy), or may be internal, e.g., protection 
against a program bug. Internal security is included in 
most operating systems. 

External security may reflect a hierarchy of author- 
ity which may be very intricate. Also, security measures 
consume system Capacity and may be very costly. Thus, se- 
curity measures should not Be imposed beyond their ee 
It has been said that the only system with perfect security 
is the one that does no useful work. 

The implementation of security is a matter of build- 
ing locks and keys. Such locks can be implemented in hard- 
ware, software or both. 

For the NPS, where the entire computer system is on 
campus, low level external security might be adequate. A 
hierarchy of authority dividing users into three classes is 

suggested: 

a. super users-those with unlimited access, usually 

|} the operators; 

b. priviledged users-those with access granted to 
most of the system resources, but subject to some constraints, 
Such as access operating system software and certain data 
bases; these users are usually members of the staff and 


faculty; 
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c. general users-those restricted to the resources 
of general use. 

The users! library should have three kinds of access 
priviledge; proprietary files with access restricted to the 
owners, read-only files with general access to read, and 
read-write files with total access allowed to anyone. The 
owners may be a group or a single person. Access priviledge 
should be established by the owner when the file is created 
and only he should be allowed to change the access priviledge. 

A simple scheme based on the use of passwords should 
be adequate. For system access and file access the user 
must specify the correct password prior to receiving access 


permission. 
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IX. PROPOSED ARCHITECTURES 


‘A. INTRODUCTION 

In this chapter four system architectures will be pre- 
sented. The degree to which each one meets the design goals 
and needs previously identified will be discussed. They will 
be presented in an evolutionary way, from the simple to com- 
plex. The aim is to disclose their strengths and weakness 
in light of the postulated design goals. These goals are 
adapted in the following way: 

l. accessibility is excluded from the consideration 
because all system architectures presented satisfy it equally; 
2. the-factor of most significance in the performance 
goal, during this initial phase of design, is graceful degra- 

dation; this will be emphasized in the discussion; 

ce Biren of use pertains almost exclusively to the 
implementation part of the design and depends on the charac- 
teristics of the interactive operating eae adopted and 
thus will be not emphasized. 

The level of abstraction of the block diagrams presented 
will be the highest possible; only those details of signifi- 
cance to the discussions will be shown. 

The existence of a batch system will be assumed, as al- 
ready mentioned; the remote job entry units (RIE), which are 
initially considered part of the batch system, will appear 
in the drawings and will be considered in the communications 


needs and in the system integration discussion. 
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1. The NPS Campus 

The NPS Campus, shown in Figure 9.1, is not large 
compared to many universities. A small region inside the 
campus contains most of the academic work. This region 
is the rectangle formed by Ingersoll Hall, Root Hall, 
Spanagel Hall, Bullard Hall and Halligan Hall. The external 
dimensions of this academic rectangle are 317 meters by 
97 meters, while the yard internal to those buildings 
measures 256 meters (Ingersoll-Spanagel) by 41 meters 
(Root=-Halligan). 

The time sharing system will be fully contained in 
the academic rectangle; needs outside this area, such as 
terminals for administrative use and for laboratories not 
included there, were not explicit and will not be consi- 
dered. Nevertheless, the extension of the system to be 
presented to other regions can be easily accomplished when 
Bnecessary because of the modularity of their architectures. 

Ingersoll Hall, where the computer center is located, 
will be referred to as the central site and will house the 
batch system and a part of the time sharing system, which 
mill vary according to the architecture. Terminals will 
also be located in this building to support its users. 

Three remote sites are identified: 

ee ehoO@e Ha Ll 

b. Spanagel Hall, and 


c. Halligan and Bullard Halls. 
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Figure 9.1 - The NPS Campus 
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Each of these remote sites will contain a remote job 
entry station (RJE) for the batch system, terminals and other 
hardware. 

Two characteristics are of importance in the defini- 
tion'of the communications facilities to be employed: 

a. system is entirely in-house, and 

b. very short communication distances. 

The best solution, considering these two factors, is 
to build NPS proprietary communications facilities for the 
new system. Physical transmission lines are the most suit- 
able technique for such short distances because of low cost, 
high reliability and absence of maintenance requirements. 

The cost of all communications links needed will be, in this 
way, a small fraction of the total cost of the system. Unlike 
geographically spread networks, minimization of communication 


cost does not have importance here. 


a ——<&x-  °i— —————— 


2. Centralized and Distributed Solutions 
: By a centralized system is meant a computer system 
based on one CPU; by contrast, distributed systems are those 
based on more than one CPU's, each performing part of the 
total work. 

The computer field was dominated since the beginning 
by the trend of centralization. With the advent of mini- 
computers this trend began to reverse and it is now clearly 
reversed. The main advantages attributed to centralization 


are generality, ease of control and economy of scale. But, 
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compared to distributed systems, they have, in general terms, 
poorer modularity, evolvability, reliability and graceful de- 
gradation. These are an important part of the goals for this 
design. Thus, the approach of this study is in favor of 
distributed systems. 

The question now is how well a distributed architec- 
pmenmatches the NPS workload. Clearly, by reasons already 
mentioned, there is the necessity of a powerful machine for 
Support of research in the NPS. It is also evident that a 
set of minicomputers cannot effectively handle many classes 
of large scientific problems that a powerful mainframe can 
handle. But two considerations have to be taken into account: 

a. the requirements for the batch computer have al- 
ready defined a large mainframe; 

b. the average work that is performed in the NPS time 
sharing system does not require large computational power; 
indeed, the ‘average user' employs a modest amount of CPU 
time per session (72 seconds for a session of 40 minutes). 

Today's minicomputer-based time sharing systems are 
being used in engineering and scientific environments with 
the same class of problems as those of NPS. These mini-~ 
computers may be adequate for handling problems similar to 
those found at NPS. 

This design is based on the principle that the batch 
system will be in charge of the more complex problems that 
require much memory and computational power. It might be 


' possible to have priviledged users working interactively on 
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this system with schedule restrictions. Most of the inter- 
active work will be performed in the distributed time sharing 


system. 


Pea RCHITECIURE L 

Figure 9.2 displays a simplified block diagram of archi- 
tecture 1, which has the following characteristics: 

1. The RJE of each remote site is directly connected to 
Pine FEP of the batch computer (BC); 

2. The time sharing computers (TSC) communicate directly 
with the BC; 

3. The TSC's are independent from the BC and are self- 
sufficient, having their own operating system, and managing 
their own system and user library; 

4. The FEP of the BC performs communications between the 
BC and TSC's or RJE's, among the TSC's and between TSC's and 
RJE's; 

5. All TSC's are identical and run under the same oper- 
ating system; 

6. In each site the number and type of the terminals are 
determined according to the local needs; 

7. TSC's may work in three modes: 

a. stand-alone time sharing; 

b. time sharing with support of the BC for some 
processes; 

c. transparent mode in which they act as bridge to 


users logged on the BC. 
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Figure 9.2 - Architecture 1 block diagram 
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This type of system is often called hierarchical, be- 
cause the system components are interconnected to form a 
tree-structured hierarchy. The capability of the components 
grow with the level which they occupy in the ‘'hierarchy'; 
at the highest level is the central computer (the BC here) 
which provides supporting services for the lower levels. 

The degree of dependence of lower levels on higher levels 
vary; for example, the central computer may provide compi- 
lation, linking and simulated testing of programs for the 
Pewee ve! Computers. in the extreme there as full de- 
pendence in a master-slave relationship. In this design the 
dependence of the TSC's on the BC was kept at a minimum due 
to reliability and graceful degradation goals. 

RJE's in this architecture coul be connected to the TSC's, 
but, being connected to the FEP, the TSC's are alleviated of 
the communications functions relative to them and the overall 
graceful degradation of the system is better. This improve- 
ment in the graceful degradation is due to the fact that the 
RJE's are used for both the time sharing and batch modes; if 
they were connected to the TSC's, the batch inputs and out- 
puts through a particular RJE would stop in the event of a 
failure in the TSC. On the other hand, the time sharing 
work is only slightly affected when an RJE does not function 
for any reason. Finally, there is no reason to connect RJE's 
to the TSC's; the TSC's would be handling the communications 


and input/output control via the RJE in this way; then the 
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use of line printers and card readers would be more cost 
effective. 

Users being allowed to log in at any public terminal poses 
a problem regarding the policy of allocation of files in the 
user library. A user may need a file which he had pre- 
viously created but which is not located in the TSC he is 
logged onto. A simple solution is to store the user library 
in the BC, but this will result in poor graceful degradation 
in the event of failure at the central site. Obviously, the 
user's library has to be distributed. Two schemes may be 
devised: static allocation and dynamic allocation. In static 
allocation the user's files have fixed residence reflecting 
the adherence of the user to a particular site. When the user 
is working outside his 'base', the files needed are shipped 
to him. At the end of his session, all the files which he 
wants to save are shipped back to his local 'residence'. 

In dynamic allocation the files migrate according to 
need; current storage is where a file was last used. The 
problem is to find a particular file; this may be done by 
storing a directory in the BC, which requires extensive 
tables, and is not good for graceful degradation purposes, 
or by having the BC poll all the TSC's until the needed file 
is located; this is practical for only a small number of TSC's. 

Because of its simplicity and taking into account the 
fact that users in the NPS tend to be localized, static 


allocation is recommended. 
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One advantage of this architecture is that the facilities 
of each site may be tuned to the specific needs of its users. 
This may seem atractive but, considering that the objectives 
of this design are a general purpose system and simplicity of 
use, this 1s not recommended beyond providing special rou- 
tines in the system library at each site. The operating 
system and the bulk of the system library must be the same 
at every site. 

The type of terminals to be employed at the lowest level 
of the nileeaenieat tree may be as diverse as desired. The 
architecture does not impose any restriction in this regard. 
On the other hand, maintenance efficiency and the simpli- 
city of use goal both call for standardization. Then the 
diversity of terminal types should be kept to the minimum 
necessary. Microprocessor based terminals with stand-alone 
capability are attractive because they may perform text and 
program editing, thus relieving the TSC of such tasks; if, 
in addition, they are coupled to floppy disk units, users 
can keep their files on diskettes, which saves space in the 
TSC disks. However, this solution has two drawbacks: it 
may not be cost effective and it does not provide good sim- 
plicity of use because users have to learn to work in two 
environments, viz, the terminal monitor and the TSC operating 
system. 

The evaluation of this architecture according to the 


design goals follows. 
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a. Modularity 
The sraend eoiene has good modularity. If the 
time sharing needs grows, the number of TSC's may increase 
accordingly. 
b. Evolvability 
Evolvability is fair. The architecture does 
not prevent the incorporation of new technology; there are 
two solutions for accomplishing this in a campus-wide, bene- 
ficial way; the first is to add the new capability to every 
Site, which may not be economical; the second is the enhance- 
ment of the BC, which is not intended for general time sharing 
use, so this is not recommended. 
G. Reliabiilrey 
Reliability is only fair. Failures in the cen- 
tral site does not affect time sharing, but one TSC failure 
puts one site out of operation. 
dad. Graceful Degradation 
Graceful degradation is poor. When one TSC fails, 
the files stored at this TSC are no longer available to users. 
Duplication of hardware at each site improves graceful degra- 
dation but is too costly. 
e. Ease of Operation 
Base of operation is poor. Operators have to 
be spread throughout the campus or have to leave the central 
Site and walk to the remote sites to fix bugs, perform ne- 


covery or start-up routines and make checks or tests. 
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A factor to consider is the division of terminal load, 
among TSC's. Since each site will have one or two TSC's, 
the number of terminals connected to them may vary from site 
to site to the point where one TSC is overloaded, while the 


other one is underutilized. 


ey ARCHITECTURE 2 

The evolution from architecture 1 to architecture 2, 
shown in Figure 9.3, is quite natural due to two factors: 

a. the remote sites have identical hardware; 

b. savings in communications bandwidth are irrelevant 

for the NPS system. 

Architecture 2 has the same logical organization as 
architecture 1. The only new element is the interconnection 
unit between the terminals and the TSC's. It provides free- 
dom of allocation of terminals to TSC's. It may be a passive 
unit and may be as simple as the patch~boards used in communi- 
cation stations. From the point of view of the user archi- 
tectures 1 and 2 are the same. But architecture 2, concen~ 
trating system resources at the central site, is much more 
advantageous in relation to the design goals: 

1. Modularity 

Modularity is very good. The system can grow in 
small increments because the TSC's are shared by all sites. 
The central position of the TSC's enable them to serve more 
than one site; consequently the total number of terminal 


lines at the central site can be divided among the sites 
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according to needs. 
eee vO Lvabi lity 
Evolvability is good. The incorporation of new 
or specialized technology in the central site may be shared 
easily by all sites; the problem is only that a new special 
processor will work with dedicated terminals. 
Seeekel ldap cy 
Reliability, according to the criterion established, 
is very good. High reliability is achieved by allocating 
an equal fraction of the number of terminals of each site 
to each TSC. The interconnection unit has a reliability 
greater than any of the other system components. 
4. Graceful Degradation 
The graceful degradation of architecture 2 is very 
good. If the TSC's are not used to their full capacity in 
terms of number of terminals, the system has an excess of 
capacity. It is worth noting that this excess serves to 
provide better response time. When a TSC fails, the corres- 
ponding terminals can be reallocated to other TSC's. User 
files are not locked in the failed TSC, because operators 
can transfer the disk drives or even the disks packs to a 
good processor. 
5. Ease of Operation 
Ease of operation is good as a result of the con- 
centration of hardware at the central site. However, many 


reconfiguration procedures have to be performed manually. 
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The division of load is another advantage of this archi- 
tecture over the previous one. In architecture 2 the TSC's 
may be allocated equally to the terminals. This can be 
referred to as statical load balancing: 

Due to the fact that at each site there are terminals 
Supported by every TSC, the user filetraffic among TSC's will 
be high because in each session a user may use a different 
processor, even at the same site. This should not be a pro- 
blem because this file transfer is done mostly at log-on and 


log-out using the statical allocation scheme. 


D. ARCHITECTURE 3 
In architecture 3, displayed in Figure 9.4, there is no 
. hierarchy. All the processors, independent of type and 
Capability, are at the same level. 

The message switch complex (MSC) is a network (or sub- 
network) of message switches (or message concentrators acting 
like switches). Each message switch has a dual configura- 
tion for reliability purposes. This is not shown in Figure 

' 9.4. The MSC uses a standard interface to all processors. 

7 In the terminal side there is no interface standard, so any 
terminal may be made. At the computer side the line speed 
is of the order of 50 to 200 kpbs, while terminal line speed 
may vary from 10 to 10,000 bps. Each message switch is typi- 
cally connected to three processors and two other switches. 
Processors are connected to more than one switch. 


The MSC has the following functions: 
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Figure 9.4 - Architecture 3 block diagram 
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a. to perform log-on and log-out procedures; 

b. to route messages from terminals to computers and 

vice-versa; 

c. to perform inter-computer communication; 

d. to route RJE-processor communication. 

With the exception of task 1, all other tasks are typical 
of those performed by a message switch. The simplicity of 
this task, however, should not burden the MSC. 

Log~on procedures may be of two types: 

a. Invisible ~ where the user does not specify the re- 
source he wants; he is automatically assigned to one of the 
general purpose TSC's; 

b. Directed ~- where the user specifies resources 
(generally the BC or special hardware). 

To avoid unnecessary system load and increase in the 
logical complexity of the MSC software, the security function 
(permission to log on) is distributed, being handled by each 
processor. 

Each processor has its own system library; except for 
special processors, it should be the same in all TSC's. The 
user library should be statically allocated for simplicity. 
The information about the residence of users' files may be 
stored in each TSC or may be concentrated in the MSC. The 
former solution is preferable in order to take the load of 


controlling file transfers out of the MSC. 
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TSC's in architecture 3 work in two modes: 

a. stand-alone time sharing; 

b. time sharing with concurrent processing, in which 
more than one processor works coordinately to provide service 
to one user. 

The evaluation of architecture 3 according to the design 
goals follows: 

1. Modularity - Modularity is very good. 

2. kEvolvability - Evolvability is very good; the stand- 
ard interface for computers in the MSC enables the addition 
of hardware to the system; also, special resources may be 
globally shared on campus. 

3. Reliability 

Reliability is good. Reliability is dominated by 
the nodes of the MSC. High reliability is achieved by dup- 
lication of hardware at each node, by connecting TSC's to 
more than one switch and by double connectivity inside the 
MSC. The price of high reliability is increased cost and 
complexity. Terminals at each site should be equally dis- 
tributed among the switches of the MSC; in this way a failure 
in one switch has equal effect on all sites but does not 
cause a particular site to stop operation. 

4. Graceful Degradation 

Graceful degradation is fair. When one switch fails, 
some of the terminals go down; reconfiguration capability 


for terminals is not present in the block diagram in Figure 
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9.4; because the number of switches is small, an interconnec- 
tion unit between terminals and the MSC is not of much help; 
this fact imposes a stringent requirement for highly reliable 
switches; it is worth noting that it is not possible to have 
an interconnection unit in parallel with the MSC because 
terminal and computer lines are not compatible; for the rest 
of the system components the graceful degradation is very 
good; failed processors are isolated by the MSC and user 
files are transferred to other processors (manually by op- 
erators or automatically by a reconfiguration unit). 

5. Ease of Operation 

Base of operation is very good. One advantage of 
this architecture is that it may provide dynamic load bal- 
ancing, i.e., users are assigned to computers in a way which 
tends to equalize the load as measured by some parameter, 
for example, the number of users on each TSC. 

One problem with architecture 3 is that the charac- 
teristics of message exchange among computers are different 
from those between terminal and computers. The conversation 
between terminal and computers tends to be done in short 
blocks while file transfer among computers is generally 
performed in large blocks. Both types are handled by the 
MSC; the file transfer can interfere badly with the inter- 
active traffic causing delays. Also, the speed of the lines 
on the computer side (50 to 200 kbps) is not high for file 


transfer. A favorable point is that the average size of 
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files is-not large in the NPS environment. One solution is 
to provide a parallel system dedicated to the intercomputer 
traffic; the MSC is then released of the task of file trans- 
fer and can be tuned to the characteristics of interactive 


era circ . 


E. ARCHITECTURE 4 

In architecture 4, shown in Figure 9.5, there is no 
subordinate ollackoasitys among processors; all are at the 
same level and are independent. They are grouped into a 
network by a loop communications Hae ieee Mee Loop Con- 
Sists of very high speed unidirectional lines and loop in- 
terface units (LI) with parallel by-pass. The capacity of 
the lines is on the order of several megabits per second; 
ror example, 10 Mbps lines may be used; in this case, a l 
Mbyte file is transmitted from one LI to its neighbor in 
800 miliseconds; thus, fast file transfer is achieved, even 
if a file has to travel through several LI's. The delay 
imposed by an LI is small because, after it recognizes that 
the file is not addressed to its processor, it may begin the 
transmission to the next LI without the need for buffering 
the entire file; in this way, error correction, eee os 
and acknowledgement are only performed at the destination LI. 
To achieve a balance between file transfer and interactive 
EGatric in the loop and also to limit the size of buffers 


in the Li's, a maximum of message block must be established. 
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Assemblage and disassemblage of messages may be handled by 
the processors. 

Due to the proximity of the processors, the loop lines 
may be implemented in parallel form; in this case the trans- 
fer speed between LI's can go to several tens of megabits 
per second. A more simple and economical way of implementing 
the loop will suffice for the NPS. Super high speed connec- 
tions are not needed because the bulk of the work is per- 
formed inside one single TSC. The use of high speed serial 
links for information transfer in parallel with low speed 
links for signaling (out-of-band sdemaling) is suggested; 
this can lead to easy LI implementation. 

The RJE's are directly connected to the FEP of the BC. 
Inputs to TSC's via RFE's are stored in the FEP and trans- 
mitted to the respective TSC through the loop upon request. 
Output of TSC's for printing are routed directly to the 
specified RJE. 

Addressing in the loop may be hard or soft. [In hard 
addressing, messages are addressed to machines, while in 
soft addressing, they are addressed to processes. Soft 
addressing has the advantage of machine independence but 
the address recognition in the LI is more involved. Hard 
addressing, on the other hand, is simple to implement and 
minimizes the message stop in LI's. For this reason hard 
addressing in the loop is preferred; in this case the mapping 


H 
function may be performed by the LI or the processor. 
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The by-pass (see Figure 9.5), in parallel with the LI's, 
serves the purpose of ‘disconnecting’ failed LI's from the 
loop. 

The interconnection unit is identical in characteris- 
tics and functions to the one of architecture 2. 

The problems of distribution of libraries and file alloca- 
tion are the same as those of architecture 3, because in both 
architectures the processors are independent. Thus, the 
solution of static allocation for user files is again recom- 
mended. The system library should, of course, be replicated 


in every TSC. TSC's have three main modes of work in this 


architecture: 
1. stand-alone time sharing; 
2. time sharing with concurrent processing; 


3. transparent mode, in which a user whose terminal is 
connected to one TSC is really logged onto another processor; 
the '‘local' TSC acts as a repeater or bridge. 

An overall evaluation of architecture 4 follows: 

1. Modularity 

Modularity is very good. Processors may be added to 
the loop as needed. Even processors not located at the cen- 
tral site may be integrated by means of an LI located there. 
While it is not recommended, the loop can be extended through- 
out the campus. 

2. Evolvability 


Evolvability is very good. Any type of processor can 
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be integrated; by the use of a standard interface, the loop 
acts as intermediary between two different processors. 

3. Reliability 

Reliability is very good. Failure in one TSC does 
not stop all the terminals of a site and does not affect the 
others. The only concern is the reliability of the loop 
links, but they have a reliability well beyond that of any 
other system component. 

4. Graceful Degradation 

The graceful degradation is very good. Terminals may 
be connected to any TSC, which enables the replacement of a 
failed processor. Failures in LI's are isolated by the by- 
pass. 

5. Ease of Operation 

Ease of operation 1s very good because of the con- 
centration of hardware at the central site. 

The simplicity of this architecture makes implementation 
relatively easy. LI's range from very simple to sophisticated 
units; this depends on the division of interface tasks between 
each processor and its LI. 

One disadvantage of architecture 4 is the possibility 
of a bottleneck 2 the loop. The Bara steniecics OE work in 
the NPS, where most of the work can generally be done on one 
TSC and where the amount of file transfer is smaller than 
interactive traffic, leads to the belief that a well designed 


loop will have a very low probability of congestion. 
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A very desirable feature of systems like those discussed 
in this chapter is the ability to perform dynamic load bal- 
ancing. It is quite possible in those systems to have one 
processor overloaded while othersare idle. Architecture 4 
may provide for dynamic load balancing by having the TSC's 
periodically exchange load information; when a new log-on 
request if completed, the TSC involved will render the 
service or will transfer the user to another processor, de- 
pending on the load equalization policy implemented. Another 
possibility is to make a slight modification of the archi- 
tecture; by replacing the interconnection unit with a paral- 
tel set of switches (message switches or PBX*s), the res- 
ponsibility of load balancing can be nelecated to the 
switching complex. This is similar to the MSC of architec- 
ture 3 with the difference that switches communicate with 
processors by terminal-like lines. This solution increases 
the cost of the system and does not have a good effect of 
reliability, but it decreases the complexity of computer 
interaction for load balancing purposes and it diminishes 
the traffic in the loop. Because of the reliability pro- 


blem it was not recommended. 


F. COMMUNICATIONS FACILITIES 
The use of physical transmission lines was previously 
recommended for the system interconnection. The suitability 


of the options available for the implementation of the commu- 
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nications facilities can be analysed in the light of the 
communications requirements. 

An approximate derivation of the requirements is per- 
formed in the following. Based on that, transmission lines 
and associated concentration techniques (for multiplexing 
or concentration) will be compared. 

The situation to be studied is that of architectures 
2 thru 4, where the remote sites have one RJE anda set of 
terminals which communicate with the central site. 

1. RJE Communication Data Rates 

The majority of the RJE's available in the market 
require communication lines of 9600 bps or less of speed. 
Since the type to be used in the NPS system has not been 
determined, an upper bound in terms of speed for RJE communi- 
cation will be derived. 

It is reasonable to assume that RJE's will be used 
for printing and card reading only and that printing speeds 
of 1000 lpm and card reading speeds of 1000 cpm are adequate 
for the remote sites. Considering the present school faci- 
lities and usage, it can be concluded that these specifica- 
tions are adequate. 

RJE maximum communication speeds are related to max- 
imum printing and reading speeds. The output transmission 
speed should be equivalent to the reading speed; if the former 
is larger than the later, line underutilization results; if 


it is slower the buffer size needed grows. Simularly input 
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speed should be matched to the printing speed in order to 
keep the input buffer size small and the line utilization 
high. A very simple estimation will be done. 

The reading speed of 1000 cpm is equivalent to appro- 
Ximately 10700 bps, considering 80 columns in a card anda 
character code of 8 bits. 

The printing speed of 1000 lpm is equivalent to 
17,600 bps, considering 132 characters in a line anda 
character code of 8 bits. 

Line protocols and error control should be added to 
those values. Since the printer will not be utilized 1003 
of the time and buffers are employed, the line speed can be 
below 17,600 bps. However, a conservative, eae Stuz0, 000 
bps will be used for this study. 

2. Terminals Data Rates 

CRT terminals are offered on the market with line 
speeds ranging from 110 to 9600 bps. 

Employing speeds of 2400 bps, a full line of 64 
characters is displayed on the screen in less than 250 mili- 
seconds, based on an overhead of line protocol and error 
control of 25%. This time is adequate since the user's 
reaction time is much slower. Thus the speed of 2400 bps 
for video terminals will be used. 

For graphics terminals a speed of 9600 bps, the 
maximum available with many graphics terminals, will be 


employed in this study. 
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3. Approximate Number of Terminals 
The model adopted for the estimation of the number of 


terminals is the multiserver queue. Often when a user goes to 
a terminal room and does not find a free terminal, he goes 
away and comes back later. The quality of such systems is 
measured by the probability that a user will not find a free 
terminal, sometimes referred as 'grade of service'. 

The computer center terminal usage data does not 
distinguish between public and private terminals. The cal- 
culation to be made is applicable to public terminals. The 
geographical distribution of users around the campus is also 
not known. Some assumptions are made to enable the calcu- 
lations: 

a. users are equally divided among the four sites 
considered; 

b. presently there are 35 public terminals; 

c. the average session on public terminals is equal 
to 40 minutes (as in the general case); 

d. the average user interarrival time for public 
terminals is 60 minutes (pessimistic because the worst case 
found for one terminal in one year was 57 minutes); 

e. poisson arrival rate; 

f. exponential service times (session time); 

g.- public terminals will be concentrated at one 


place at each site. 
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Following these assumptions, the overall traffic 
rate in Erlangs is: 
40 


where M is the number of terminals and p the utilization 
factor for terminals. Then, in one site the traffic rate 
is 24/4=6. 


The probability that no terminal is free is given by: 


(Mp)" 
os M! 
B N 
= (Mp) 
N=0 N! 
Fixing P, at 0.01 and using Mp=6 the required number 


B 


of public terminals at one site is 13. 

The number of private terminals is then assumed to 
increase to 20 and to be equally divided among the sites. 

Then at each site there will be 18 terminals. Among 
these it 1s assumed there are 12 CRT terminals and 6 graphics 
terminals. 

Confidence in these results is obviously poor be- 
cause of the lack of data. The numeric assumptions made were 
all biased in favor of increasing the number of terminals. 
These values can then be used to estimate communications needs. 

4. Total Bandwidth Required at Each Remote Site 

One remote site will require on the average: 


a. 1 RJE (20,000 bps); 
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b. 12 CRT terminals (2400 bps each); 

c. 6 graphics terminals (9600 bps each). 

The total data rate at one remote site is then 106,400 
bos. 

5. Communications Techniques Options 

Considering the distances involved and the communica- 
tions requirements derived above, the options for communica- 
Eions are: 

a. TBM ancd. fiber optics; 

be [DM and coaxial cable; 

c. modems coupled to telephonic FDM transmitting 
through a twisted pair; 

d. multipair cable of twisted pairs; 

e. TDM and twisted pairs. 

Options a. and b. are equivalent; the choice is 
between fiber optics and coaxial cable. There are TDM equip- 
ment available in the market that go to 1.54 Mbps well above 
the requirement of 106 Kbps. 

In option c. voice FDM is used in connection with two 
twisted pairs (equalization needed). In each voice channel 
a modem is connected and serves one device at the site. For 
example, if a 24 channel FDM multiplexer is used, 18 modems 
are required for terminals (2400 and 9600 bps) and one high 
speed modem of 19.2 kbps (higher speeds are available on the 
market) is connected to one or more voice channels (for example, 


to a 12 KHZ channel) as required. Thus 21 channels are utilized 
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and 3 are free. The advantage of FDM is its modularity (not 
found in TDM), but this option is more expensive than the 
onegyengic 

Option d. is the most simple of the solutions. Con- 
Sidering the fact that the greatest distance involved is 256 
meters ina straight line, it is possible to connect directly 
the terminals to the computers by means of multi-twisted pair 
cable (telephonic cable). Allowing for non-straight paths 
in ducts and wall climbing it may be assumed that, in the 
worst case, the distance is 750 meters. ie ene cerminais 
and computer ports employ the differential transmission 
method of the RS-422 balanced interface, the maximum rate 
through a single 24 AWG twisted pair at 750 meters is 105 
Kbps. Then, employing RS-422, terminals and RJE's can be 
directly connected to the central site by means of two 
twisted pairs for each device. The problem is that RS-422 
is not widely marketed; in some cases the more popular RS-232 
direct connection is possible (depending on duct distance). 

In option e. TDM equipments employing RS-422 trans- 
mit through one twisted pair. 

The final choice depends on many factors. An over- 
all ranking of the options is more a function of cost and 


practical considerations than of technology. 
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X. CONCLUSION 


In the first part of this work the principles of computer 


——_— 


networks were exposed and analysed. 


In the second part, a study was conducted with the objective 


a 
eee = a 


i 


of presenting alternative solutions for a new time sharing 
system for the Naval Postgraduate eoncoll based on distributed 
systems. An analysis of the characteristics of the computing 
work at the NPS was performed. Quantitative and qualitative 
information was derived from the data on the usage of the 
CP/CMS system at the school. Based on these factors, design 
goals were formulated. Four system architectures were dis~— 
cussed in the light of the design goals. It was shown that 
these distributed systems achieve the goals better than cen- 
tralized systems. Many other particular points of interest 
of each architecture were discussed. Those architectures are 
not necessarily tied to minicomputer implementation. Medium 
scale computers may have advantages in terms of capability 
and satisfaction of the complex requirements. 

An overall evaluation of the four architectures depends 
on additional factors which were not considered such as cost, 
administrative policies and market availability. Purely in 
light of the principles formulated, architectures 2 and 4 
are recommended. If simplicity of implementation is stressed, 
architecture 2 is the solution. If generality and flexibil- 


ity are most important architecture 4 should be adopted. 
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